General motion estimation and segmentation from image sequences.

This thesis is concerned with the problem of motion estimation and segmentation, mainly related to planar motion in the image plane. The emphasis is placed on several important issues, namely: the study of different motion models and their performance, the benefits resulting from the use of contextual information, the application of multiresolution strategies, and the use of Robust methods and confidence measures. The thesis investigates the application of global motion models, in particular the affine model, in different estimation and segmentation approaches. It is shown that the use of such models, which globally constrain the estimate, results in improved accuracy and robustness. Robust techniques, which can cope with outliers often present when larger data sets are used, are adopted and tested here. The performance is further improved by the use of confidence measures, and of contextual information such as intensity edges or moving feature information. Two broad classes of approach are developed and investigated. The first one is based on the theory of Markov Random Fields. Novel elements in this approach include the introduction of a complex motion model - capable of describing translation, rotation and change of scale - and confidence factors describing the reliability of the data. The application of the Supercoupling approach for multiresolution optimisation speeds up convergence and further improves the quality of the estimate. The second class of algorithms is based on the Hough Transform. An in-depth investigation of the behaviour of the standard Hough Transform is conducted. This leads to the adoption of a robust statistics method providing a better estimate accuracy, better motion segmentation and guaranteed convergence. The use of multiresolution representation in the image plane, in addition to multiresolution in the parameter space, brings the advantage of robust and fast convergence even for large displacements. An important contribution of the research is the evaluation of different kernel functions from the point of view of robustness to noise and change in illumination conditions. Two algorithms from this group have been developed. The first one processes an entire image and provides parallel motion segmentation and estimation. The other is used as a local and robust method for the estimation of optic flow, with the ability to detect multimodal motions. A comparative study with other state-of-the-art methods is conducted, and the results are strongly in favour of the new algorithms. In summary, all stages of motion estimation and segmentation have been investigated. At the low-level, a robust algorithm for optic flow estimation has been developed. It can cope with multiple moving objects, and detects motion boundaries and occluded/uncovered regions. The spatial coherence of motion is enforced here very strongly, resulting in an accurate estimate and reliable confidence measures. This low-level estimate may be globally interpreted, together with other clues and a priori knowledge of the world using a multi-scale Markov Random Field approach. Alternatively, motion estimation and segmentation may be performed in parallel globally using the Robust Hough Transform approach. At this stage meaningful objects can be segmented, thus providing a high-level description of the scene