Occlusion detection in multi-baseline stereo

Identifying occlusion is a challenging problem for general stereo algorithms. There have been some efforts to detect occlusion regions. In these approaches, occlusion detection is either implicitly incorporated into the matching process or is invoked as a post processing step after initial disparity estimation. However, the conventional methods of treating occlusion have limitations in that they do not explicitly find evidence of occlusion. As a result, occlusion boundaries are often poorly localized. This in turn, results in blurred disparity estimates around depth discontinuities, which obscures important information such as object boundaries. We propose a novel method to detect occlusions and occluding contours. Instead of inferring occlusion from disparity estimates, we try to explicitly find occlusions from confirming evidence. We use the fact that occlusions generate photometric inconsistencies and we search for decorrelations in the disparity space image. We present an analysis of the structure of the disparity space image which utilizes its duality with the epipolar plane image. Our occlusion analysis also allows us to eliminate incorrect matches by exploiting the structure of these disparity space images. From the detected occlusion regions, we extract occluding contours which are aligned with the edges of foreground objects. In this thesis, we also describe how the sparse disparity and discontinuity estimates produced by the proposed method can be used to accurately reconstruct a dense disparity map