3D facial expression recognition based on multi-view and prior knowledge fusion

Abstract This paper presents a novel multi-view convolutional neural network (CNN) model for 3D facial expression recognition (FER). In contrast to existing deep learning-based 3D FER approaches that mainly learn the expressions from frontal facial attribute images, the proposed model incorporates multi-view and facial prior information of the observed 3D face into the learning process. This information is jointly trained in an end-to-end manner to predict the emotion of the input 3D face model. The experiments on public 3D facial expression datasets show that training the CNN with additional information from different views and facial prior knowledge would result in learning more discriminative features as against from a single view. Our model outperforms the state-of-the-art 3D FER methods in term of recognition accuracy indicating its effectiveness. Moreover, the improvement of the proposed model is displayed more clearly in the discrimination of low-intensity facial expressions