Micro-expression recognition analysis using facial strain / Liong Sze Teng

Facial micro-expression analysis has attracted much attention from the computer vision and psychology communities due to its viability in a broad range of applications, including medical diagnosis, police interrogation, national security, business negotiation, and social interactions. However, the micro and subtle occurrence that appears on the face poses a major challenge to the development of an efficient automated micro-expression recognition system. Therefore, to date, the annotation of the ground-truths (i.e., emotion label, onset, apex and offset frame indices) are still performed manually by psychologists or trained experts. This thesis briefly reviews the conventional automatic facial microexpression recognition methods and their related works. In general, an automatic facial micro-expression recognition system consists of three basic steps, namely: image preprocessing, feature extraction, and emotion classification. This thesis mainly focuses on the enhancement of the first two steps over conventional methods in the literature. Specifically, a hybrid facial regions selection for pre-processing is proposed. This method is able to eliminate some parts of the face that are irrelevant to any facial emotions. Then, an effective feature descriptor, namely, optical strain, is utilized to capture the variations in characteristics and properties of the micro-expressions in the video. Next, a feature descriptor is developed to encode the essential expressiveness of the apex frame because the information of a single apex frame exhibits the highest variation of motion intensity, which is adequate to represent the emotion of the entire video. Finally, this thesis is concluded by highlighting its contributions and limitations, as well as suggesting possible future directions related to micro-expression recognition system