Do you speak to a human or a virtual agent? automatic analysis of user’s social cues during mediated communication

International audienceWhile several research works have shown that virtual agents are able to generate natural and social behaviors from users, few of them have compared these social reactions to those expressed during a human-human mediated communication. In this paper, we propose to explore the social cues expressed by a user during a mediated communication either with an embodied conversational agent or with another human. For this purpose, we have exploited a machine learning method to identify the facial and head social cues characteristics in each interaction type and to construct a model to automatically determine if the user is interacting with a virtual agent or another human. The results show that, in fact, the users do not express the same facial and head movements during a communication with a virtual agent or another user. Based on these results, we propose to use such a machine learning model to automatically measure the social capability of a virtual agent to generate a social behavior in the user comparable to a human-human interaction. The resulting model can detect automatically if the user is communicating with a virtual or real interlocutor, looking only at the user's face and head during one second