Facial feature detection and tracking with a 3D constrained local model

This thesis establishes a framework for facial feature detection and human face movement tracking. Statistical models of shape and appearance are built to represent the human face structure and interpret target images of human faces. The approach is a patch-based method derived from an earlier proposed method, the constrained local model (CLM) [1] algorithm. In order to increase the ability to track face movements with large head rotations, a 3D shape model is used in the system. And multiple texture models from different viewpoints are used to model the appearance. During fitting or tracking, the current estimate of pose (shape coordinates) is used to select the appropriate texture model. The algorithm uses the shape model and a texture model to generate a set of region template detectors. A search is then performed in the global pose / shape space using these detectors. Different optimisation frameworks are used in the implementation. The training images are created by rendering expressive 3D face models with different scales, rotations, expressions, brightness, etc. Experimental results are demonstrated by fitting the model to image sequences with large head rotations to evaluate the performance of the algorithm. To evaluate the stability and selection of factors of the algorithm, more experiments are carried out. The results show that the proposed 3D constrained local model algorithm improves the performance of the original CLM algorithm for videos with large out-of-plane head rotations