Changing our perception of communication in virtual environments

When people communicate face-to-face they use gestures and body language that naturally coincide with speech [McNeill, 2007, Gesture Thought, University of Chicago Press.]. In an immersive virtual environment (VE) we can control both participants' visual feedback of self and the other in order to investigate the effect of gestures on a communication task. In our experiment the communication task is to make the listener say a word without the speaker saying the word. We use animated real-time self-avatars in immersive VEs to answer the question: `Does the use of naturalistic gestures help communication in VEs'. Specifically, we perform a within-subject experiment which investigates the influence of first- and third-person perspectives, and of animated speaker and listener. We find that people significantly perform better in the communication task when both the speaker and listener have an animated self-avatar and when the camera for the speaker shows a third-person perspective. When participants moved more they also performed better in the task. These results suggest that when two people in a VE are animated they do use gestures to communicate. These results demonstrate that in addition to the speaker movements, the listener movements are important for efficient communication in an immersive VE