Motion estimation and segmentation of colour image sequences

The principal objective of this thesis is to develop improved motion estimation and segmentation techniques that meet the image-processing requirements of the post¬production industry. Starting with a rigorous taxonomy of existing image segmentation techniques, we proceed by focusing on motion estimation by means of optical flow calculation. A parametric motion model based method to estimate optical flow fields on three consecutive frames is developed and tested on a number of colour real sequences. Initial estimates are robustly refined in an iterative scheme and are enhanced by colour probability distribution information to enable foreground/background segmentation in a maximum a posteriori pixel classification scheme. Experiments, . show the significant contribution of the colour part towards a well-segmented image.Additionally, a very accurate variational optical flow computation method based on brightness constancy, gradient constancy and spatiotemporal smoothness constraints is modified and implemented so that it can robustly estimate global motion over three consecutive frames. Motion is enhanced by colour evidence in a similar manner and the method adopts the same probabilistic labelling procedure. After a comparison of the two methods on the same colour sequences, a third neural network based method is implemented, which initially estimates motion by employing two twin-layer optical flow calculating Gellular Neural Networks and proceeds in a similar manner, (incorporating colour information and probabilistic ally classifying pixels), leading to similar or improved quality results with the added advantage of significantly accelerated performance. Moreover, another CNN is employed with the task of offering spatial and temporal pixel compatibility constraint support, further improving the quality of the segmented images. Weights are used to control the respective contributing terms enabling optimization of the segmentation results for each sequence individually. Finally, as a case study of CNN implementation in hardware (FPGA), the use of Handel-G, a C-like, high-level, parallel, hardware description language, is exploited to allow for rapid translation of our algorithms to efficient hardware