Fall 2004 ECE598YM Constraints on Multiple Apparent Contours and Edge Feature Based 3d Reconstruction

Apparent contours or silhouettes contain information about the object’s geometry and camera motions. Silhouettes alone can be used to recognize objects [3] or estimate camera motions [2]. Furthermore, 3d shape of an object can be easily approximated from silhouettes by computing the Visual Hull [1]. However, extracting silhouettes is difficult and human intervention is often necessary. There are some approaches that automatically finds silhouette[4, 5], but their algorithms are restricted to simple scenes: for instance, black object and grey background. Of course, if you can control the environment, silhouette extraction can be easier. However, in general situations where an object and background have textures, extracting contour is a very difficult task. Initial motivation of this project is to answer the following question: if you have multiple fully-calibrated images and need to extract silhouettes of an object, can we make use of the constraints on multiple contours and make contour extraction easy? Relationships of multiple apparent contours are well studied and it is known that they must satisfy certain equations, which is quadratic in the number of total images. We can even recover the 3d coordinates of points on the surface from these constraints. However, a brief answer to the above question turns out to be no. Those constraints do not really help in terms of extracting contours, because intensity edges in images that are not from occluding boundaries but merely from textures, behave exactly the same. In other words, we cannot avoid being trapped by edges of textures which we do not want. However, this fact, on the other hand, leads to a novel approach of point correspondence and 3d reconstruction. The algorithm and its results are presented in the following sections. In this report, first, we quickly review the constrains of multiple apparent contours and see why they do not help in extracting contours. Then second, we provide a novel 3d reconstruction algorithm by using the analysis of the first step, and finally show some results