Prefrontal Control Over Occipital Responses to Crossmodal Overlap Varies Across the Congruency Spectrum

While matched crossmodal information is known to facilitate object recognition, it is unclear how our perceptual systems encode the more gradual congruency variations that occur in our natural environment. Combining visual objects with odor mixtures to create a gradual increase in semantic object overlap, we demonstrate high behavioral acuity to linear variations of olfactory–visual overlap in a healthy adult population. This effect was paralleled by a linear increase in cortical activation at the intersection of occipital fusiform and lingual gyri, indicating linear encoding of crossmodal semantic overlap in visual object recognition networks. Effective connectivity analyses revealed that this integration of olfactory and visual information was achieved by direct information exchange between olfactory and visual areas. In addition, a parallel pathway through the superior frontal gyrus was increasingly recruited towards the most ambiguous stimuli. These findings demonstrate that cortical structures involved in object formation are inherently crossmodal and encode sensory overlap in a linear manner. The results further demonstrate that prefrontal control of these processes is likely required for ambiguous stimulus combinations, a fact of high ecological relevance that may be inappropriately captured by common task designs juxtaposing congruency and incongruenc