Motion-based part segmentation, shape and motion estimation of multi-part objects

Computer vision has begun to play all increasingly important role in applications like anthropometry, human factors design, ergonomics, teleconferencing and virtual reality as well as performance measurement of both athletes and patients with psycho-motor disabilities. All of these areas require the determination of the parts of a multi-part object (e.g., the human body) and/or the estimation of their shape and motion parameters. This thesis focuses on methods for segmentation as well as shape and motion estimation of multi-part objects in general and of humans in particular. We present an analytical technique (Part Segmentation Algorithm) which, without using a prior model, recovers all the moving parts of a multi-part object by monitoring and reasoning over the changes of the object\u27s deforming apparent contour. To improve the quality of shape estimation, we present a novel technique for the automatic spatio-temporal adaptation of the values of a deformable model\u27s elastic parameters that are used in our physics-based method. This technique obviates the need for manual initialization of these parameters and results in superior fitting results. We apply the framework described above to the problem of acquiring a subject\u27s three dimensional body model. We first use the PSA to segment the apparent body contour of a moving subject into its constituent parts. Then, the three dimensional shape of a subject\u27s body parts is estimated by fusing information from images taken from three cameras placed orthogonally. Our technique applies to humans of any anthropometric dimension. Next, we introduce a formal framework for the three-dimensional model-based motion capture and animation of unconstrained human movement from multiple cameras. The advantage of our tracking technique is that the subject does not have to wear markers or other special equipment in order to estimate his/her motion. Finally, we present part-segmentation, tracking and animation results (and a performance analysis) from a series of experiment involving the recovery of shape and three-dimensional motion parameters in the presence of significant occlusion