Benefits of multisensory presentation on perception, memory and navigation [De voordelen van multisensorische presentatie op perceptie, geheugen en navigatie]

The overarching purpose of this thesis was to investigate the beneficial effects of congruent visual, auditory, and tactile presentation on navigation performance. Overall, the results show that multisensory presentation is advantageous. An important aspect of this finding is that multisensory performance was always equal to or higher than the best unisensory 102 Chapter 8 performance. Therefore, we expect that presenting multisensory information in virtual environment applications has only positive effects. Of course, the cost of implementing multisensory environments (e.g. investment) should be lower than the benefits (e.g. reduction of training time; return on investment). While we report positive effects of multisensory presentation in all chapters of this thesis, we have only conducted a single experiment which shows that multisensory presentation actually improves navigation. It is yet unclear to what extent the effects of this experiment will carry over to a wider range of virtual environmental applications. Further research is required to provide adequate guidelines when and when not the implementation of (congruent) multisensory presentation is advisable. We have two comments relevant to this issue. Firstly, the investigation of multisensory presentation in virtual environments in this thesis was limited to local landmarks (i.e. landmarks that are only observable from their immediate surroundings). Many more elements in a virtual environment can be made multisensory and the benefits thereof need not necessarily be the same to those for multisensory local landmarks. Global landmarks (i.e., landmarks that are observable from any location within a large area; e.g. a church tower), environments (e.g. urban or rural), and displacement (e.g. footsteps and/or engine sounds) can all easily be presented congruently in audition and vision. Displacement could also be presented to touch. Even if memory and navigation benefits for these elements are not as large as those we report for local landmarks, the benefits may all add up when everything is presented congruently in multiple sensory modalities. A second and final consideration for the implementation of congruent multisensory presentation is that sensory channels only have a limited bandwidth. This means that if, for example, one chooses to use auditory and/or tactile modalities to present information complementary and congruent with the visual modality - and the auditory and tactile resources are used to the full extent - that the auditory and tactile modalities cannot be used to convey other and perhaps more beneficial information. For instance, the presentation of auditory information may interfere with our ability to communicate. On the other hand, multisensory presentation gives the navigator the option to ad hoc use a sensory modality for other purposes without fully reducing navigation performance. In conclusion, our experiments show that multisensory presentation benefits perceptual, memory, and navigation performance. In addition, our experiments have provided further insight in the mechanisms underlying multisensory interactions. They imply that the presentation of multisensory information in virtual environments may help to overcome some of the current difficulties associated with navigating virtual environments. We expect that multisensory presentation increases a navigator’s ability to learn information about the environment and helps him or her to select better routes, thus improving the navigator’s ability to find his way in a virtual environment