Studying social interactions through immersive virtual environment technology: virtues, pitfalls, and future challenges

The goal of the present review is to explain how immersive virtual environment technology (IVET) can be used for the study of social interactions and how the use of virtual humans in immersive virtual environments can advance research and application in many different fields. Researchers studying individual differences in social interactions are typically interested in keeping the behavior and the appearance of the interaction partner constant across participants. With IVET researchers have full control over the interaction partners, can standardize them while still keeping the simulation realistic. Virtual simulations are valid: growing evidence shows that indeed studies conducted with IVET can replicate some well-known findings of social psychology. Moreover, IVET allows researchers to subtly manipulate characteristics of the environment (e.g., visual cues to prime participants) or of the social partner (e.g., his/her race) to investigate their influences on participants' behavior and cognition. Furthermore, manipulations that would be difficult or impossible in real life (e.g., changing participants' height) can be easily obtained with IVET. Beside the advantages for theoretical research, we explore the most recent training and clinical applications of IVET, its integration with other technologies (e.g., social sensing) and future challenges for researchers (e.g., making the communication between virtual humans and participants smoother). Humans spend between 32 and 75% of their waking time in social interactions (Mehl and Pennebaker, 2003). To understand how we behave in social interactions, how we draw conclusions about our social interaction partners, or how the outcome of the social interaction will shape us and our social relationships, we need to observe and study humans engaged in a wide variety of different social contexts. Given the frequency of its occurrence and the importance of social interactions for understanding humans and for bringing about change for individuals and society, the lack of research using direct behavioral observation is surprising (Baumeister et al., 2007). One reason for this gap is that if we focus on natural observation, we may have to wait long periods of time before a desired social situation occurs naturally with us being present to observe it. In an attempt to overcome these constraints, researchers typically use simulations, meaning that people are put in a specific social situation from which their behavior is observed and the interaction outcomes assessed. In the present review, we describe how such simulations can take place in an immersive virtual environment (IVE) with virtual humans as social interaction partners and we discuss the distinct advantages and challenges of this method. In this article, we focus on social interactions with virtual humans in the IVEs and their use for research and training. While IVET has been around for several decades, the use of this technology for the social sciences is still relatively new (Fox et al., 2009) and particularly the aspect of including virtual humans as social interaction partners to simulate interpersonal encounters is still in its infancy. It is the latter aspect on which we will shed light by describing the state of the art in this domain, some of the main findings, and the existing challenges and future directions of this line of research. Our contribution is at the same time an update of the earlier review by Fox et al. and a focalization on the simulation of social interactions with virtual humans