What is the origin of cross-modal correspondences?

There are many seemingly arbitrary associations between different perceptual properties across modalities, such as the frequency of a tone and spatial elevation, or the color of an object and temperature. Such associations are often termed crossmodal correspondences, and they represent a hallmark of human and animal perception. The pervasiveness of crossmodal correspondences, however, is at odds with their apparent arbitrariness: why encoding arbitrary mappings across sensory attributes in such a consistent manner? Aren’t they misleading unless they represent some fundamental properties of the world around us? Over the last few years a number of studies have demonstrated that crossmodal correspondences are not arbitrary at all: they faithfully represent the statistics of natural scenes, which can be learnt over time and exploited to better process multisensory information. Here, we provide an overview of such most recent evidence, with a particular emphasis on the mapping between auditory pitch and spatial elevation, a celebrated case of crossmodal correspondence whose apparent arbitrariness has baffled neuroscientists for more than a century. By combining a direct measurement of environmental statistics with bioacoustics, psychophysics, and Bayesian modeling, we have recently shown that such mapping is not only already present in the environment: it is also directly encoded in the bioacoustics, that is, in the shape of the human outer ear. Taken together, current evidence calls for a thorough characterization of the environmental statistics as the most critical step towards a comprehensive understanding of the origins and the roles of cross-modal correspondences in perception, cognition, and action