On the use of Optical Flow for Scene Change Detection and Description in Outdoor Lighting-variant Environments

For a mobile robot to perform autonomous tasks it is essential that it can determine its location in the environment before it can navigate and carry out an assigned mission. Research on autonomous vehicle localization and its use in navigation has therefore been a significant research topic in robotics for more than two decades. Detection of key locations in indoor and corridor-like environments has been investigated thoroughly mainly using sonar sensors and laser range finders. Computer vision has also been used for the detection of these locations with limited results, and most approaches are only valid in indoor scenes with controlled lighting. One popular group of methods is based on image appearance. However, these methods have a shortcoming in outdoor environments in that they are not very tolerant to illumination changes, harsh lighting, and shadows. An alternative approach, that is highly robust to challenging lighting conditions, is the use of optical flow. There is also good evidence for the use of optical flow by insects for the purpose of odometry and it has been reported that honeybees use changes in the magnitude of the induced optical flow to detect and identify landmarks near food sources. Â The aim of the research reported in this thesis is to examine to what extent optical flow alone can be exploited to extract and describe the environment from a ground vehicle. Using optical flow information only, the goal is to detect and describe unique and salient locations in outdoor urban environments. These key locations form the nodes in a topological mapping and localization framework. The connections (edges) between the topological nodes are described by the odometer path. The odometer information is further used to normalize the magnitude of the induced optical flow vectors and to remove any contributions from rotational motions of the vehicle. In this research, the statistical characteristics of the optical flow signal are analyzed and it is demonstrated how robust flow vectors can be generated and used for detection of scene changes in the form of depth discontinuities and appearance changes at key locations in the environment. To successfully achieve this task, a comprehensive discussion of camera positioning, distortion compensation, noise filtering, and parameter estimation is presented. The repeatability of the proposed method for detecting scene changes in indoor and outdoor environments in challenging and diverse lighting conditions is demonstrated by conducting a large number of trials, both indoors and outdoors. When a scene change is detected, the induced optical flow information is used to generate a descriptor of that scene. Two schemes, based on the shape context descriptor and on lower and higher order statistical moments of the optical flow information, are presented for generation of these descriptors. Once a database of nodes and their corresponding optical flow descriptors is created, topological localization is performed by matching the current frame to the database to estimate the most likely position in the environment. For matching the optical flow moment descriptors the Mahalanobis distance is used, while for the optical flow shape context descriptors the Chi-square distance is used. In addition to the study on optical flow-based scene change detection and description, an approach to visual odometry for car-like vehicles traversing urban environments is presented. It is demonstrated that, by using multiple-templates the reliability of the correlation matching is increased and by introducing a template quality measure (for estimating the suitability of a template for the purpose of correlation) and a linear forward prediction filter (for restricting the search area) real-time operation is achievable. The visual odometer is successfully implemented on two different vehicles and on different surfaces, including concrete, asphalt, gravel and off-road grassy terrain. The applicability of the odometer for high-speed traversal is demonstrated by showing the results of test conducted at driving speeds exceeding 30km/h in suburban environments. The method presented in this thesis for detection and description of scene changes, contrasts to image descriptor-based techniques, in being robust to strong variations in lighting. This is verified by comparing the presented method to a state-of-the-art bag-of-words appearance-based loop closure scheme using SIFT descriptors. The repeatability of the presented method is demonstrated through the large number of real-world trials