The strong role of contour geometry in Structure-from-motion and its implications for projective consistency

Structure from motion (SFM) studies have shown that observers can perceive 3D structure in dynamic dot displays that are projected from rotating 3D objects. The SFM literature has focused on explaining how human observers ‘reconstruct’ the 3D structure from dynamic dot displays. Some analyses are based on the positions of dots, others are based on the velocity field. For a long time, there have been two implicit assumptions in SFM research: first, the perceived 3D structure should be projectively consistent with the 2D display; second, the computation of 3D percept does not depend on contour geometry, only on the motion profile of the dots. However, there have been some suggestions in the literature that neither of the two assumptions is correct (Ramachandran, Cobb, & Rogers-Ramachandran, 1988; Thompson, Kersten, & Knecht, 1992; Froyen, Feldman, & Singh, 2013; Tanrıkulu, Froyen, Feldman, & Singh, 2016). Under certain conditions, the perceived 3D structure is not projectively consistent with the 2D motion, and in those cases, the 3D percept is generally dominated by the contour geometry (despite the inconsistency with the motion). Nevertheless, the role of con- tour geometry in SFM remains largely ignored, and none of the existing SFM models (in both Psychology and Computer Science) incorporate its role. The current dissertation aims to pro- vide more direct evidence against these two assumptions and investigate possible connections between them. In chapter 1, we overview the literature in the field of human structure-from-motion research. The studies on SFM have been either based on the position of individual dots, or on the analysis of motion profile. Both types of studies assume that the perceived 3D structure should be projectively consistent with the 2D display. Also, they seem to ignore the role of contour geometry in the determined 3D structures. We show that some studies have provided evidence that questions the above two assumptions. The aim of current dissertation is to explore both topics, and its potential connections. Our hypothesis is that both motion and contour geometry play important roles in SFM, and when the two sources of information conflict with each other, the influence of contour geometry can be strong enough to override the inconsistency in motion profile. Therefore, this may provide an explanation for why projective consistency is not a necessary requirement in SFM. In chapter 2, we manipulate the dots’ 2D speed profile in symmetric asymmetric and parallel shapes using a method introduced in He (2016), which allows us to systematically control the degree of projective consistency in the displays. The latter two types of shape contain projective inconsistency both in the speed profile and contour geometry. However, our results show that the volumetric percept with asymmetric displays is still vivid, providing strong evidence that projective consistency is not a necessary requirement in SFM. The similar strength of the 3D percept between symmetric and asymmetric displays also suggests that the global contour symmetry may not be as important as we thought in SFM. In these displays, the speed profile is locally consistent with contour geometry along each horizontal cross-section, so the zero-speed at the boundary suggests a slant of 90◦, which may serve as a cue to be an occluding contour of a 3D object. It suggests that instead of global geometry, we should probably focus on the local presence of 2D contour, which may itself provide a cue to the occluding contour of a 3D object, even if the speed profile does not go down to 0 there. In chapter 3, we investigate the role of contour geometry by manipulating it separately from the speed profile of the dots. Specifically, we manipulate the shape of the aperture through which the same image motion is shown. Because of the absence of any occlusion cues, the shape of the aperture essentially defines the bounding contour of the SFM display. We start with a ‘motion region’ (such as rectangle, trapezoid, hexagon, barrel, diamond, ellipse) containing dot motion consistent with 3D rotation, and then transform the shape of this motion region to define a smaller aperture. In many conditions, rather than looking like dots moving behind an aperture, the contour captures and defines the perceived 3D shape. The question is how much the aperture shape can be transformed and still determine the 3D percept (i.e. contour-defined percept which is now projectively inconsistent with the image motion). We used Method of Constant Stimuli to find the thresholds for the contour-defined percept. In Experiment 1, we generated smaller apertures by compressing the horizontal width of the motion region. We found that when the horizontal width ratio ω was as low as 0.2 (ellipse) or 0.6 (rectangle), the SFM percept was still dominated by the contour shape. This shows the importance of contour geometry in SFM. In Experiments 2 and 3, we broke coaxiality (the aperture shape and motion region no longer shared the same axis) by either translating or rotating the aperture within the motion region. We found that observers were quite sensitive to the deviation from co-axiality. In Experiment 4, we introduced variation of horizontal width ratio ∆ω within the same display (such as a rotating sphere display shown through a triangle). We found no systematical effect of ∆ω. However, the results did show systematic effects of convexity and contour smoothness. Overall: 1) These studies provide strong evidence that projective consistency is not a necessary requirement in SFM. Observers can readily perceive a 3D shape even when it is not projectively consistent with the SFM display; 2) These studies document the prominent role played by contour geometry in SFM, as well as some of the ways in which contour geometry and image motion interact in SFM. The prominent influence of contour geometry also in part explains why SFM percepts can fail to be projectively consistent with image motion. Rather than ignore contour geometry (as the SFM literature has largely done), the current study highlights the need for further systematic investigations into how contour geometry and image motion combine in SFM to generate a 3D percept. This in turn would make it possible to develop mathematical models that incorporate both of these cues in SFM.Ph.D.Includes bibliographical reference