Spatial and temporal prediction schemes for object-based digital video

grantor: University of TorontoObject-based video compression schemes have recently gained tremendous interest, and are the driving force of the future MPEG 4 and 7 video coding standards. These object-based coders not only provide for a higher compression efficiency, but they can also support content-based functionalities for future multimedia applications. In this thesis, spatial and temporal prediction techniques are investigated for these newer object-based compression schemes. Structural and statistical models are first investigated for spatially interpolating digital images for editing/compositing purposes in object-based video or for frame-based compression systems. Nonlinear filter structures based on vector order statistics are selected for spatially interpolating color images due to their robustness, preservation of edge information and image details, and their ability to exploit the interchannel color correlations. Statistical methods based on Gibbs random field (GRF) models are also used to obtain an interpolated image. The iterative GRF methods are approximated by a non-iterative nonlinear filtering operation, thereby reducing the computational complexity of the process. The proposed methods are all compared to the conventional linear approaches. Temporal prediction of videophone-type sequences is subsequently investigated in view of the problems associated with conventional block-based methods. Segmentation-based motion compensated prediction is carried out by partitioning an image into a facial region and a set of arbitrarily-shaped regions. A novel and useful approach is presented to automatically locate and track the facial area within image sequences using the visual cues of color and shape. The scheme is robust in scenes with head-and-shoulders type images found in videophone sequences or facial image databases, and can be easily tuned for more general cases where many faces may be present in the scene. A general color segmentation technique is then proposed to partition the remainder of the image and form a triangular mesh model. This is subsequently followed by a suitable mesh tracking scheme. Motion compensated prediction is finally performed by utilizing the estimated nodal point motion vectors and an affine warping transformation. Experimental results provide a comparison between the proposed scheme and the conventional block-based methods.Ph.D