Multimodal 2D+3D face recognition

The growing interest in face recognition is fueled by its potential applications in commercial and law enforcement. Although humans seem to have an innate ability to recognize and distinguish between faces, computerized face recognition systems have not yet been able to achieve satisfactory performances. The worldwide implementation of face recognition has been hindered by its inherent problems. The appearance of a face may vary significantly due to variations in illumination condition, head pose, and facial expression, to name a few. On the other hand, faces basically belong to the same class of objects. They consist of the same facial features in the same geometrical configuration. When all other factors are held constant, faces of different persons are somewhat similar and the differences between them can be quite subtle. Face recognition has been regarded as one of the fundamental problems in pattern analysis. In the past, research efforts have been focused on face recognition approaches based on 2D intensity images. However, their performances deteriorate under illumination and pose variations. There is evidence that 3D range images have the potential to overcome this problem. The explicit representation of a 3D shape is less sensitive to illumination and pose variations. However, the 3D data is vulnerable to surface deformation due to facial expression variations. Integrating 2D and 3D sensory information is believed to potentially improve the face recognition performance. Most of the proposed approaches of multimodal 2D+3D face recognition exploit the 2D and 3D information at the later stage. They do not fully benefit from the dependency between the 2D and 3D images. Furthermore, early fusion data contains richer information about the biometrics than the compressed features or matching scores. Therefore, I propose PCA- and FLD-based image recombination approaches to enrich the distinctive information contained in the images and to enhance the discriminating power of the recombined data. The PCA recombines the 2D and 3D images using decorrelation axes that account for the maximal amount of variances in the data. Meanwhile, the FLD calculates an optimum recombination transform that maximizes the ratio of the determinant of the between-class and within-class scatter matrices of the recombined data. Therefore, the recombined data is more discriminating for recognition than the original images.Doctor of Philosophy (EEE