Nonparametric facial feature localization

Any facial feature localization algorithm needs to incor-porate two sources of information: 1) prior shape knowl-edge, and 2) image observations. Existing methods have primarily focused on different ways of representing and in-corporating the image observations into the problem so-lution. Prior shape knowledge, on the other hand, has been mostly modeled using parametrized shape models. Parametrized shape models have relatively few parameters to control the shape variations, and hence their represen-tation power is limited with the examples provided in the training data. In this paper, we propose a novel method for modeling the prior shape knowledge. Rather than using a holistic approach, as in the case for parametrized shape models, we model the prior shape knowledge as a set of local com-patibility potentials. This “distributed ” approach provides a greater representation power as it allows for individual landmarks to move more freely. The prior shape knowl-edge is incorporated with local image observations in a probabilistic graphical model framework, where the infer-ence is achieved through nonparametric belief propagation. Through qualitative and quantitative experiments, the pro-posed approach is shown to outperform the state-of-the-art methods in terms of localization accuracy. 1