Accurate landmarking of three-dimensional facial data in the presence of facial expressions and occlusions using a three-dimensional statistical facial feature model

Abstract—Three-dimensional face landmarking aims at auto-matically localizing facial landmarks and has a wide range of applications (e.g., face recognition, face tracking, and facial ex-pression analysis). Existing methods assume neutral facial expres-sions and unoccluded faces. In this paper, we propose a general learning-based framework for reliable landmark localization on 3-D facial data under challenging conditions (i.e., facial expres-sions and occlusions). Our approach relies on a statistical model, called 3-D statistical facial feature model, which learns both the global variations in configurational relationships between land-marks and the local variations of texture and geometry around each landmark. Based on this model, we further propose an occlu-sion classifier and a fitting algorithm. Results from experiments on three publicly available 3-D face databases (FRGC, BU-3-DFE, and Bosphorus) demonstrate the effectiveness of our approach, in terms of landmarking accuracy and robustness, in the presence of expressions and occlusions. Index Terms—Facial expression, fitting, landmarks, occlusion, statistical face model, 3-D face feature. I