Amodal multimodal integration

Recently it has been shown that congruent visual and haptic signals are integrated in a statistically optimal fashion. Spatial separation between the signals can preclude this integration. Here we investigate whether optimal integration occurs between an amodally completed visual stimulus and its haptic counterpart. Thus, we ask whether integration occurs despite the sensory information not being derived from the same spatial location. This may indicate that subjects inferred that the visually specified parts of the stimulus and the haptic information have a common cause and thus should be integrated. The visual stimulus was a disparity-defined bar that was partially occluded (amodal completion condition). The bar could also be touched behind the occluder using two fingers. Subjects' task was to discriminate the size of two successively presented bars using a 2-IFC paradigm, where one interval contained conflicting haptic and visual information. Performance in the amodal completion condition was not different from a condition in which the occluder was removed (visual-haptic condition). Both conditions were consistent with an optimal integration strategy. More interestingly, integration deviated from optimality when we introduced a slight modification to the visual stimulus - small gaps between the bar and the occluder (gap condition). This manipulation interfered with the amodal completion process and consequently subjects relied almost completely on the haptic information for discriminating the size of the bars. These findings suggest that visual and haptic information can be combined optimally even when visual information is not directly specified by sensory information, but results from amodal completion. In conclusion, it seems that the perceptual system determines when to combine visual and haptic information based on the likelihood the signals have of belonging to the same object (i.e. if there is a causal relationship between the signals) and not only on signal co-location