From Emotions to Action Units with Hidden and Semi-Hidden-Task Learning

Limited annotated training data is a challenging prob-lem in Action Unit recognition. In this paper, we investigate how the use of large databases labelled according to the 6 universal facial expressions can increase the generaliza-tion ability of Action Unit classifiers. For this purpose, we propose a novel learning framework: Hidden-Task Learn-ing. HTL aims to learn a set of Hidden-Tasks (Action Units) for which samples are not available but, in contrast, train-ing data is easier to obtain from a set of related Visible-Tasks (Facial Expressions). To that end, HTL is able to ex-ploit prior knowledge about the relation between Hidden and Visible-Tasks. In our case, we base this prior knowl-edge on empirical psychological studies providing statisti-cal correlations between Action Units and universal facial expressions. Additionally, we extend HTL to Semi-Hidden Task Learning (SHTL) assuming that Action Unit training samples are also provided. Performing exhaustive exper-iments over four different datasets, we show that HTL and SHTL improve the generalization ability of AU classifiers by training them with additional facial expression data. Addi-tionally, we show that SHTL achieves competitive perfor-mance compared with state-of-the-art Transductive Learn-ing approaches which face the problem of limited training data by using unlabelled test samples during training. 1