The Effects of Multisensory Cues on the Sense of Presence and Task Performance in a Virtual Reality Environment

The aim of this study was to evaluate the effect of visual, haptic and audio sensory cues on participant’s sense of presence and task performance in a highly immersive virtual environment. Participants were required to change a wheel of a (virtual) racing car in the 3D environment. Subjective ratings of presence and comfort were recorded using the Immersive Tendencies Questionnaire (ITQ), [WS98], the Presence Questionnaire (PQ) [WS98] and Simulator Sickness Questionnaire (SSQ), [KLB*_93]. The time taken to complete the task was used as an objective performance measure. Auditory, haptic and visual cues signalling critical events in the simulation were manipulated in a factorial design. Participants wore 3D glasses for visual cues, headphones for audio feedback and vibration gloves for tactile feedback. Participants held a physical pneumatic tool. Events, such as the full extraction of a bolt were signalled by haptic (vibration frequency change), acoustic (change in tool sound) and visual (colour change of bolt) cues or combinations of cues. Data was collected in two blocks containing all eight sensory cue combinations: the task was once performed in a normal VR environment (control) and once (motion) in an environment where the position of the virtual environment was sinusoidally modulated by 2 cm in the depth plane at 0.5 Hz to simulate inaccurate participant tracking. All participants completed all 16 conditions in a pseudorandom sequence to control for order and learning effects. Subjective ratings for presence, discomfort and perceived cues effectiveness were recorded after each condition. Participants performed best when all cues were present. Significant main effects of audio and tactile cue presentation on task performance and also on participants' presence ratings were found. We also found a significant negative effect of environment motion on task performance and participants' discomfort ratings