The dual role of culture on signalling and receiving dynamic facial expressions

Human survival critically relies on communicating a broad set of social messages including physical states and mental states. The prerequisite for any successful social communication is the shared knowledge between the sender and the receiver about what and how a specific social signal is used. To communicate the broad set of social messages in daily life, human beings have developed complex facial movement patterns as one of the most important and powerful social signals. With increasing globalization, cross-cultural interactions are fast becoming integral to modern life, which presents increasing pressure for cross-cultural communication. Specifically, a broad set of facial expressions including conversational facial expressions is critical for clear communication because they guide the flow of social exchanges. Yet, our knowledge of such facial movement patterns is relatively limited in terms of their functions in different cultural context – for example, whether these important everyday facial expressions are understood across cultures or cause cross-cultural confusions. In this thesis, I explored how facial movement patterns are used in Western and East Asian culture to communicate a broad set of social messages including physical states and mental states. Specifically, I objectively characterized the structure of dynamic facial expression patterns using 4D computer graphics and a data-driven social psychophysics method. I then examined the role of culture in signaling and receiving facial expressions using the signal detection theory and Mutual Information analysis. Together, my results reveal for the first time how specific facial movement patterns are used to communicate a broad set of social messages in Western and East Asian culture and how culture shapes the signalling and perception of such facial expressions in cross-cultural communication. Finally, I discussed the implication of my results in the field of psychology, computer science and social robotics, with links to my future work on developing a mathematical model of face social signalling and transfer this knowledge to socially and culturally sensitive conversational agents