How sounds shape visual processing: acoustic modulation of primary and higher visual cortices

Current understanding posits that the different senses interact on multiple stages of processing. We investigated this notion using a combination of high-resolution fMRI and electrophysiological recordings in non-human primates. In particular we study how acoustic stimulation affects neural responses in low-level (areas V1 and V2) and higher-tier visual areas (area TE). In primary visual cortex we find that the presentation of sounds reduces the fMRI-BOLD signal to values below pre-stimulus baseline, but enhances evoked response to visual stimuli above the level attained by pure visual stimulation. This finding is well consistent with previous results in humans, and suggests a modulatory and gain changing mechanism of acoustic stimuli on visual cortex. Noteworthy, the acoustic influence was stronger in peripheral visual representations, in good concordance with prior reports on anatomical projections from auditory areas to peripheral representations in occipital cortex. Similarly, we find that acoustic stimuli modulate neural responses in visual temporal association cortex (area TE). Recordings of neural activity in alert animals demonstrate a modulatory influence on neural responses, but only minimal responses to acoustic stimulation alone. Using computational tools to characterize this acoustic influence we find that the presence of sounds enhances the trial to trial reliability of neural responses and increases the ability to correctly decode which of a set of visual scenes was presented. This result hence suggests that sounds increase the encoded visual information