Recovering Camera Motion and Mobile Objects in Video Documents

International audienceThis chapter presents a set of methods related to the recovery of camera motion parameters and to the segmentation of mobile objects in video documents for content indexing. This includes methods for the segmentation of video documents into continuous shots, methods for motion analysis, methods for extracting reliable trajectories within shots, and two different methods for the recovery of the camera motion (relatively to the main background), the first one for a camera maintained at a fixed location with rotational and zoom degrees of freedom, and the second one for a camera of arbitrary motion but assuming a fixed focal length. The first camera motion recovery method is based on the search of an optimal projective transform between consecutive images combined with an iterative background / mobile objects segmentation process. The second one is based on a paraperspective factorization method for shape and motion recovery. The presented methods are illustrated in the context of a video indexing system developed at CLIPS-IMAG into which they are integrated. The system also attempts to classify shots or sub-segments of shots into one of the following categories of ''no motion'', ''non mobile camera motion'', ''mobile camera motion'' or ''other type of motion''. Further sub-categorization can be done for each recovered type. Sample results are presented using sequences extracted from video documents of the ISIS GDR-PRC GT10/AIM test corpus