Capturing complex spatio-temporal relations among facial muscles for facial expression recognition

Spatial-temporal relations among facial muscles carry crucial information about facial expressions yet have not been thoroughly exploited. One contributing factor for this is the limited ability of the current dynamic models in cap-turing complex spatial and temporal relations. Existing dy-namic models can only capture simple local temporal rela-tions among sequential events, or lack the ability for incor-porating uncertainties. To overcome these limitations and take full advantage of the spatio-temporal information, we propose to model the facial expression as a complex activity that consists of temporally overlapping or sequential primi-tive facial events. We further propose the Interval Temporal Bayesian Network to capture these complex temporal re-lations among primitive facial events for facial expression modeling and recognition. Experimental results on bench-mark databases demonstrate the feasibility of the proposed approach in recognizing facial expressions based purely on spatio-temporal relations among facial muscles, as well as its advantage over the existing methods. 1