Automatic Face Landmarking in 3D.

Face analysis from 3D scanned data is an important alternative to the more common approaches that exploit information from 2D images and video. The long standing challenges of illumination and pose variance do not affect 3D data in the same measure. The newly available shape information in the form of 3D surfaces has already been shown to improve the performance of conventional 2D data systems in the important task of face recognition. Facial landmarking is part of many recognition approaches. For example, canonical face models are fitted onto the data by using key landmark points, statistical procedures require a detailed initial marking procedure and face segmentation for expression analysis is aided by using landmark locations. Despite being the common denominator for many automatic 3D face systems, landmarking of faces is currently done manually or semi-automatically. However, manual landmarking is not only a time consuming task but is also prone to inaccuracies. This thesis proposes a system that accurately finds several face landmarks simultaneously in faces with or without expressions. The system that we refer to as Automatic Landmarking of Faces in 3D, ALF3D, models two of the main characteristics of landmarks that can be observed on 3D data, shape and location. ALF3D consist of two statistical models, one for shape and one for location, that cooperate together to find the locations of a set of facial landmarks. The location model maintains the relative location of landmarks whilst the shape model searches for that point the best matches the expected shape of each landmark. Experiments show that automatic landmarks found with ALF3D are on average between 3mm and 5mm from manually labelled landmarks. It is also demonstrated that the use of automatic landmarks can improve the performance of a face recognition system. An additional benefit obtained with ALF3D is that it finds a large set of landmarks including several that have not been studied before and that are key for expression analysis. This thesis also introduces an automatic expression classification approach for faces in 3D. The procedure explores the idea that landmarking of faces in 3D can be performed more accurately with a prior knowledge of the face’s expression. To this purpose the concept of Deformed Shape Profiles (DSP) is introduced. A DSP captures the deformations due to expressions of local regions around landmarks. The automatic expression classification system is shown to be accurate and to significantly improve facial land-marking of faces with expression