Robust real-time detection, tracking, and pose estimation of faces in video streams

Human face analysis has been recognized as a crucial part in intelligent systems. However, there are several challenges before robust and reliable face analysis systems can be deployed in real-world environments. One of the main difficulties is associated with the detection of faces with variations in illumination conditions and viewing perspectives. In this paper we present the development of a computational framework for robust detection, tracking and pose estimation of faces captured by video arrays. We discuss development of a multi-primitive skin-tone and edge-based detection module integrated with a tracking module for efficient and robust detection and tracking. A multi-state continuous density Hidden Markov Model based pose estimation module is developed for providing an accurate estimate of the orientation of the face. A systematic evaluation of these algorithms and the overall framework is performed with an extensive set of experiments. Results of these experiments suggest the validity of the proposed framework and its computational modules. 1