A Multi-Method Approach for Discriminating Between Similar Facial Expressions, Including Expression Intensity Estimation

Facial expression provides sensitive cues about emo-tion and plays a major role in interpersonal and human-computer interaction. Most facial expression recognition systems have focused on only six basic emotions and their concomitant prototypic expressions posed by a small set of subjects. In reality, humans are capable of producing thou-sands of expressions that vary in complexity, intensity, and meaning. To represent the full range of facial expression, we developed a computer vision system that automatically recognizes individual action units (AUs) or AU combina-tions using Hidden Markov Models and estimates expres-sion intensity. Three modules are used to extract facial ex-pression information: (1) facial feature point tracking, (2) dense flow tracking with principal component analysis (PCA), and (3) high gradient component detection (i.e. fur-row detection). The average recognition rate of upper and lower face expressions is 85 % and 88%, respectively, using feature point tracking, 93 % (upper face) using dense flow tracking with PCA, and 85 % and 81%, upper and lower face respectively, using high gradient component detection. 1