Research Article Neural Synergy Between Kinetic Vision and Touch

ABSTRACT—Ambiguous visual information often produces un-stable visual perception. In four psychophysical experiments, we found that unambiguous tactile information about the direction of rotation of a globe whose three-dimensional structure is am-biguous significantly influences visual perception of the globe. This disambiguation of vision by touch occurs only when the two modalities are stimulated concurrently, however. Using func-tional magnetic resonance imaging, we discovered that touching the rotating globe, even when not looking at it, reliably activates the middle temporal visual area (MT1), a brain region com-monly thought to be crucially involved in registering structure from motion. Considered together, our results show that the brain draws on somatosensory information to resolve visual conflict. People’s daily activities are guided by an amalgam of sensory inputs from different modalities. These sensory modalities, although typically segregated in textbooks, function together to specify behaviorally important objects and events. To give just a few examples, sound and vision interact to influence speech perception (McGurk & MacDonald, 1976) and to specify the nature of dynamic events such as collision (Sekuler, Sekuler, & Lau, 1997). Similarly, sound can influence the perceived roughness of a touched surface (Guest, Catmur, Lloyd, & Spence, 2002), and touch can influence visual perception of surface texture (Heller, 1982) and surface slant (Ernst, Banks, & Bulthoff, 2000). In the work reported here, we sought to extend the analysis of multimodal perception to an aspect of visual perception—structure from motion—about which there is some evidence concerning possi-ble underlying neural mechanisms. In particular, we exploited the kinetic depth effect: the perception of a three-dimensional (3D) object on the basis of differential optic flow (Doner, Lappin, & Perfetto, 1984; Wallach & O’Connell, 1953). When viewing the 2D parallel projection of a rotating 3D object, one may experience spontaneous reversals in the perceived direction of rotation (Fisichelli, 1947; Howard, 1961