Local Learning to Improve Bag of Visual Words Model for Facial Expression Recognition

In this paper we propose a novel computer vision method for classifying human facial expression from low resolution images. Our method uses the bag of words representa-tion. It extracts dense SIFT descriptors ei-ther from the whole image or from a spa-tial pyramid that divides the image into in-creasingly fine sub-regions. Then, it repre-sents images as normalized (spatial) presence vectors of visual words from a codebook ob-tained through clustering image descriptors. Linear kernels are built for several choices of spatial presence vectors, and combined into weighted sums for multiple kernel learning (MKL). For machine learning, the method makes use of multi-class one-versus-all SVM on the MKL kernel computed using this rep-resentation, but with an important twist, the learning is local, as opposed to global – in the sense that, for each face with an unknown la-bel, a set of neighbors is selected to build a local classification model, which is eventually used to classify only that particular face. Empirical results indicate that the use of presence vectors, local learning and spatial information improve recognition performance together by more than 5%. Finally, the pro-posed model ranked fourth in the Facial Ex-pression Recognition Challenge, with an ac-curacy of 67.484 % on the final test set