Behind the Depth Uncertainty: Resolving Ordinal Depth in SFM

Abstract. Structure from Motion(SFM) is beset by the noise sensitivity problem. Previous works show that some motion ambiguities are inher-ent and errors in the motion estimates are inevitable. These errors may render accurate metric depth estimate difficult to obtain. However, can we still extract some valid and useful depth information from the inac-curate metric depth estimates? In this paper, the resolution of ordinal depth extracted from the inaccurate metric depth is investigated. Based on a general depth distortion model, a sufficient condition is derived for ordinal depth to be extracted validly. By studying the geometry and statistics of the image regions satisfying this condition, we found that although metric depth estimates are inaccurate, ordinal depth can still be discerned locally if physical metric depth difference is beyond cer-tain discrimination threshold. The resolution level of discernible ordinal depth decreases as the visual angle subtended by the points increases, as the speed of the motion carrying the depth information decreases, and as points recede from the camera. These findings suggest that accurate knowledge of qualitative 3D structure is ensured in a small local image neighborhood, which might account for biological foveated vision and shed light on the nature of the perceived visual space.