Recognizing Lower Face Action Units for Facial Expression Analysis

Most automatic expression analysis systems attempt to recognize a small set of prototypic expressions (e.g. hap-piness and anger). Such prototypic expressions, however, occur infrequently. Human emotions and intentions are communicated more often by changes in one or two discrete facial features. In this paper, we develop an automatic sys-tem to analyze subtle changes in facial expressions based on both permanent (e.g. mouth, eye, and brow) and tran-sient (e.g. furrows and wrinkles) facial features in a nearly frontal image sequence. Multi-state facial component mod-els are proposed for tracking and modeling different facial features. Based on these multi-state models, and without artificial enhancement, we detect and track the facial fea-tures, including mouth, eyes, brow, cheeks, and their related wrinkles and facial furrows. Moreover we recover detailed parametric descriptions of the facial features. With these features as the inputs, 11 individual action units or action unit combinations are recognized by a neural network al-gorithm. A recognition rate of 96.7 % is obtained. The recognition results indicate that our system can identify ac-tion units regardless of whether they occurred singly or in combinations. 1