On Decomposing an Unseen 3D Face into Neutral Face and Expression Deformations

Abstract. This paper presents a technique for decomposing an unseen 3D face under any facial expression into an estimated 3D neutral face and expression deformations (the shape residue between the non-neutral and the estimated neutral 3D face). We show that this decomposition gives a robust facial expression classification and improves the accuracy of an off-the-shelf 3D face recognition system. The proposed decomposition system is a multistage data-driven process in which training expression residues and neutral faces reciprocally guide the decomposition of the 3D face. A plausible decomposition was achieved. The shapes and the nor-mals of the expression residue are used for expression classification while the neutral face estimates are used for expression robust face recognition. Experiments were performed on a large number of non-neutral scans and significant expression classification rates were achieved. Moreover, 6 % in-crease in face recognition rate was achieved for probes with severe facial expressions.