Cross-modal perception of actively explored objects

Abstract. Many objects in our world can be picked up and freely manipulated. thus allowink information about an object lo be available to both the visual and haptie symeks. However, we understand very little about how object information is shared across the modalities. Under constrained viewing cross-modal object recognition is most etlicient \\,hen the same surface of an object is presented to the visual and haptic systems 151. Here \\.e tested cross modal recognition of novel objects under active manipulation and unconstrained \,iewing ofthe objects. These objects were designed such that each surface of the object provided unique information. In Experiment 1. participants were allowed 30 seconds lo learn the objects visually or haptically. Haptic learning resulted in relatively poor haptic recognition performance relative to visual recognition. In Experiment 2, we increased the learning time for haptic exploration and found equivalent haptic and visual recognition, but a cost in cross modal reeognition. In Experiment 3, participants learned the obiects using both modalilies together, vision alone or haptics alone. Recognition performance was tested using both modalities together, We found that recognition performance \\,as signilicantly bettcr when objects were lcarned by both modalities than either of the modalities alone. Our results suggest that efficient cross modal performance depends on the spatial correspondence of object surface information across modalities.