3D Global Head Pose Estimation: A Robust Approach

We present a generic and robust method for 3D model-based global head pose estimation in monoscopic, and non-calibrated video sequences. The proposed method relies on a 3D/2D matching between 2D image features estimated throughout the sequence and 3D object features of a generic head model. Providing accurate 3D pose estimates is constrained by: 1) selecting a relevant set of sequence features together with associated robust estimators; 2) defining an appropriate reference head model; 3) quantifying the sensitivity of the matching process with respect to the model initialization. The developed estimation method combines motion and texture features in an iterative optimization procedure based on the downhill simplex algorithm. A proper initialization of the model pose, based on a block matching procedure, is performed at each frame in order to take into account large amplitude head motions. For the same reason, we developed a non-linear optical flow-based interpolation algorithm for increasing the frame rate. Experiments demonstrate that this method is stable over extended sequences including large head motions, occultations, various head postures and lighting variations. The estimation accuracy is related to the head model, as established by using an ellipsoidal model and an ad hoc synthesized model. The method proposed is general enough to be applied to other tracking applications