Face recognition under varying illumination, pose and contiguous occlusion

This thesis considers the problem of recognizing human faces despite variations in illumination, pose and contiguous occlusion, using only frontal training images. In particular, we are interested in simultaneously handling multiple modes of variability in automatic face recognition. We first propose a very simple algorithm, called Nearest-Subspace Patch Matching, which combines a local translational model for deformation due to pose with a linear subspace model for lighting variations. This algorithm gives surprisingly competitive performance for moderate variations in both pose and illumination, a domain that encompasses most face recognition applications, such as access control. The results also provide a baseline for justifying the use of more complicated face models or more advanced learning methods to handle more extreme situations. We further develop a more principled and general method for face recognition with contiguous occlusion using tools from sparse representation, which has demonstrated promising results in handling illumination changes and occlusion. While such sparsity-based algorithms achieve their best performance on occlusions that are not spatially correlated (i.e. random pixel corruption), we show that they can be significantly improved by harnessing prior knowledge about the pixel error distribution. We show how a Markov random field model for spatial continuity of the occlusion can be integrated into the computation of a sparse representation of the test image with respect to the training images. Our algorithm efficiently and reliably identifies the corrupted regions and excludes them from the sparse representation. Extensive experiments on publicly available databases verify the efficacy of the proposed methods