Toward affective social interaction in VR

Giulio Jacucci shares his views on affective social interaction in virtual reality (VR). In social interaction, emotions can be expressed through gestures, posture, facial expressions, speech and its acoustic features, and touch. The expressions are recorded first by capturing the live presentation from a professional actor, using a facialtracking software that also animates a virtual character. Expressions can then be manually adjusted to last for the same amount of time and end with a neutral expression. Emotion tracking is more challenging in the case of a wearable VR headset, as facial expressions cannot be easily tracked through recent computervision software. Eye tracking can be used both to identify whether users attend to a particular stimulus to track its emotional response, and to track psychophysiological phenomena such as cognitive load and arousal. Affective interaction here can motivate physical exercise or monitor psychophysiological states such as engagement or relaxation.Non peer reviewe