Multisensory integration: From human behavior to neural systems

To interact effectively with our environment, the human brain integrates information from multiple senses. While multisensory integration was traditionally assumed to be deferred until later processing stages in higher order association cortices, more recent studies have revealed multisensory integration even in putatively unisensory cortical areas. Given this multitude of multisensory integration sites, characterizing their functional similarities and differences is of critical importance. Combining functional imaging (fMRI), effective connectivity analyses and psychophysics in humans, our studies highlight three main aspects: First, the locus of multisensory integration depends on the type of information being integrated and the specific relationship between the auditory and visual signals. Second, in terms of functional brain architectures, effective connectivity analyses suggested that audiovisual interactions in low level sensory areas are mediated by multiple mechanisms including feedforward thalamocortical, direct connections between sensory areas and top down influences from higher order association areas. Third, the regional response profile and activation patterns depend on the relative reliability of the unisensory signals. Paralleling behavioural indices of multisensory integration, multivariate pattern analyses revealed that multisensory integration increased the discriminability and hence reliability of multisensory representations already at the primary cortical level. From the macroscopic perspective of regional BOLD signals, our data provide further evidence for ‘Bayesian-ish’ integration of signals from multiple senses