Lighting-and Personal Characteristic-Aware Markov Random Field Model for Facial Image Relighting System

In this study, we proposed an example-based facial image relighting framework to relight an unseen input 2D facial image from a specified source lighting condition to a target lighting. Such a relighting framework is highly challenging, since the highlight or shadow areas in the relighted image usually follow the specific facial feature characteristics of the input image. When facial images have to be lighted in a specified lighting condition style followed a few provided pairwise lighting examples, the problem becomes even more complex. In contrast to existing learning-based relighting framework that directly predicts the intensity of the target image, we use a personal and lighting-specific transformation to formulate the appearance correlation between source and target lighting conditions. We propose a lighting-and personal characteristic-aware Markov Random Field Model to estimate the transformation parameter of the input subject, which is integrated with additional classifiers to determine input facial regions should be enhanced or maintained. Experimental results show that the proposed kernel facial relighting model can avoid overfitting problem and generates vivid and recognizable results despite the scarcity of training samples. Furthermore, the relighted results successfully simulate the individual lighting effects produced by the specific personal characteristics of the input image, such as the nose and cheek shadows. Finally, the effectiveness of the proposed framework is demonstrated using a robust face verification test for images taken under side-light conditions.補正完