Model-Based Real-Time Head Tracking

Abstract|This paper treats real-time tracking of a human head using an analysis by synthesis ap-proach. The work is based on the Structure from Motion (SfM) algorithm from Azarbayejani and Pentland [3]. We will analyze the convergence prop-erties of the SfM algorithm for planar objects, and extend it to handle new points. The extended algo-rithm is then used for head tracking. The system tracks feature points in the image using a texture mapped three-dimensional model of the head. The texture is updated adaptively so that points in the ear region can be tracked when the user's head is ro-tated far, allowing out-of-plane rotation of up to 90± without losing track. The covariance of the x ¡ and the y¡coordinates are estimated and forwarded to the Kalman ¯lter, making the tracker robust to oc-clusion. The system automatically detects tracking failure and reinitializes the algorithm using informa-tion gathered in the original initialization process