Structure-from-motion: Perceptual evidence for surface interpolation

AbstractDynamic random-dot displays representing a rotating cylinder were used to investigate surface interpolation in the perception of structure-from-motion (SFM) in humans. Surface interpolation refers to a process in which a complete surface in depth is reconstructed from the object depth values extracted at the stimulus features. Surface interpolation will assign depth values even in parts of the object that contain no features. Such a “fill-in” process should make the detection of featureless stimulus areas (“holes”) difficult. Indeed, we demonstrate that such holes in our rotating cylinder can be as wide as one-quarter of the stimulus before subjects can reliably detect their presence. Subjects were presented with a variation on the rotating cylinder in which all dots were oscillating either in synchrony or asynchronously. Subjects perceive a rigidly rotating cylinder even when such a percept is not in agreement with the physical stimulus. To reconcile this discrepancy between actual and perceived stimulus we propose that individual points contribute to a surface based object representation and that in this process the visual system looses access to the identity of the individual features that make up the surface. Finally we are able to explain a variety of previously documented perceptual peculiarities in the perception of structure-from-motion by arguing that the perceptual interpretation of the object's boundaries influences the surface interpolation process. These findings offer strong perceptual evidence for a process of surface interpolation and are also physiologically plausible given results from recordings in awake behaving monkey cortical areas V1 and MT. The companion paper demonstrates how such a surface interpolation process can be incorporated into a structure-from-motion algorithm and how object boundaries can influence the perception of structure-from-motion as has been demonstrated before and in this paper