The computer analysis of facial expressions : on the example of depression and anxiety

Significant advances have been made in the field of computer vision in the last few years. The mathematical underpinnings have evolved in conjunction with increases in computer processing speed. Many researchers have attempted to apply these improvements to the field of Facial Expression Recognition (FER). In the typical FER approach, once an image has been acquired, possibly from capturing frames in a video, the face is detected and local information is extracted from the facial region in the image. One popular approach is to build a database of the raw feature data, and then use statistical measures to group the data into representations that correspond to facial expressions. Newly acquired images are then subjected to the same feature extraction process, and the resulting feature data compared to that in the database for matching facial expressions. Academic studies tend to make use of freely available, annotated sets of images. These community databases, used for training and testing, are usually built from acted or posed expressions [Kanade 00, Wallhoff] of primary or prototypical emotion ex-pressions such as fear, anger and happiness. Making use of video or images captured in a natural setting is less common, and fewer studies attempt to apply the techniques to more subtle and pervasive moods and emotional states, such as boredom, arousal, anxiety and depression. This dissertation aims to test whether state-of-the-art developments in the field of computer vision can be successfully applied in a practical situation involving non-primary FER. The functional requirements of a system that can perform full lifecycle, video analysis of vocal and facial expressions are outlined. These have been used to build a fully-functional prototype system that incorporates Active Appearance Model (AAM)s. The system has been integral to supporting the experimental aspects of this dissertation. Of particular interest in this dissertation is the recent evolution in computer vision of the AAM. These are used to locate fiduciary, or landmark, points, around a face in an image. If the landmark points can be reliably and consistently found within an image then the collective "shape" for the points, together with the pixel information, can be used to build representations of facial expressions. Two experiments were undertaken and are reported in this thesis. The first investigated whether FER practices could be applied to sense for anxious expressions in images. The second was designed to analyse the facial activity and expressions in video recordings of patients diagnosed with Major Depressive Disorder (MDD). Finally, the practical limitations of the statistical approach to FER are considered along with strategies for overcoming those limitations