Facial Beauty Analysis based on Computer Vision and Deep Learning Techniques

Analysis of facial beauty has become an emerging research topic in recent years, and has fascinated researchers from various fields. Considering the development of modern technologies, there is a great interest in discovering the quantitative relationship between attractiveness and facial features using computer vision and machine learning techniques. The objective of facial beauty prediction (FBP) is to develop a human-like model that automatically evaluates facial attractiveness. Our research aims to provide new frameworks to analyze the attractiveness of human faces. First, two geometric facial measurements, including ratios and angles, along with a stacking ensemble model are employed to predict female face attractiveness. In fact, we investigate the importance of angles of lines connecting facial landmarks on three prominent facial regions, namely eyes, nose and chin, for facial beauty assessment. Second, inspired by the success of Convolutional Neural Networks (CNNs) on facial analysis tasks, we introduce a new framework to analyze the attractiveness of female faces using transfer learning methodology as well as a stacking ensemble model. Specifically, a pre-trained Convolutional Neural Network (CNN) originally trained on relatively similar datasets for face recognition task is utilized to acquire high-level and robust features of female face images. This is followed by leveraging a stacking ensemble model which combines the predictions of several base models to predict the attractiveness of a face. Extensive experiments conducted on SCUT-FBP and SCUT-FBP5500 benchmark datasets, confirm the strong robustness of the proposed approach. Prediction correlations of 0.89 and 0.91 are achieved by our new method for SCUT-FBP and SCUT-FBP5500 datasets, respectively. Our successful results would certainly support the efficacy of transfer learning when applying deep learning techniques to compute facial attractiveness. Moreover, this study provides a new deep framework for simultaneous facial beauty assessment, gender recognition as well as ethnicity identification. To further enhance the attractiveness computation accuracy, specific regions of face images (i.e. left eye, nose and mouth) as well as the whole face are fed into multi-stream CNNs. To the best of our knowledge, this is the first work to integrate different face attributes and facial regions using multi-stream CNNs for face attractiveness prediction. Prediction correlation of 0.95 is achieved by our deep model for the SCUT-FBP5500 benchmark dataset, which would certainly support the efficacy of our proposed model. Interestingly, our experimental results on the SCUT-FBP and SCUT-FBP5500 benchmark datasets (spanning 500 and 5500 facial images) indicate significant improvement in accuracy over the other state-of-the-art methods