Differential techniques for the accurate estimation of image flow

One of the prime objectives of many robotic systems is the ability to judge depth for self navigation. This depth or range information in a three-dimensional scene has traditionally been obtained from point correspondences in static two-dimensional images, a process often known as photogrammetry. This approach has the major difficulty that point pairs corresponding to a common three-dimensional scene point are not easy to identify automatically in images. In more recent years a wide range of alternative approaches for estimating depth have emerged. One such group of techniques makes use of multiple images from a single moving sensor. Known as shape from motion algorithms, they often employ a three step process to estimating depth. The first step involves estimating the optical or image flow. The second step then estimates the global motion parameters of the image sensor from the image flow. The final step involves estimating the relative depth from the image flow and motion parameters. Thus in order to make reliable depth estimates, accurate image flow measures are required. This thesis, therefore, concentrates on the detailed analysis of techniques employed for accurately determining image flow. Preliminary chapters introduce basic computer vision concepts and image flow algorithms. As some of the terminology used in this field is ambiguous, particular care has been placed on presenting a well defined set of terms, notably defining: "camera motion", the "motion field", "optical flow" and "image flow". Also defined is the "aperture problem" which describes a fundamental inability to perceive or measure motion in directions were no change in the image intensity function is present. It is demonstrated that all techniques rely on the presence of non-zero second order differential terms of the image sensor intensity function to overcome the aperture problem. Following chapters present a review and comparison of a number of image flow techniques, with particular emphasis placed on gradient methods. It is argued that the fundamental problem of measuring image flow is to determine the velocity in as small an aperture as possible. Based on this criteria it is concluded that the technique based on a weighted least squares solution of first order differential terms of the image intensity is the best generalized second order method for achieving this objective. The practical issues of regularization of the gradient calculations and the influence of noise corruption are investigated. Conclusions drawn show the importance of smoothing and the effects of excessive smoothing, including a loss of spatial precision and an increase in numerical error due to the computation of small gradient terms. Importantly it is demonstrated that spatial and temporal gradients are not independent and consequentially their respective errors are correlated. It is thus shown that many image flow estimation techniques exhibit a systematic bias. An investigation into methods of assessing the quality of image flow estimates is also made. Indicators of the quality of an estimate are required in order to evaluate the accuracy of subsequent estimates made from them; particularly depth estimates, segmentation of the image flow field, and determining the optimal aperture size for making image flow estimates. Findings illustrate that the correlation between many commonly used, or proposed, indicators of image flow error are poor. Some investigative work into two innovative approaches for estimating image flow, based on the use of colour and actual camera motion parameters, is presented. It is shown that both colour and camera motion parameters provide an additional constraint to the estimation procedure, which can be used to improve the resolution of the image flow field by allowing smaller apertures to be considered. Preliminary results from these studies suggest that both sources of additional information improve the robustness of the algorithms. The final part of this thesis presents some quantitative and qualitative results for image flow estimates from real imagery. A summary of conclusions is presented and a number of advanced issues and suggested extensions are outlined