Learning to Find Occlusion Regions

For two consecutive frames in a video, we identify which pixels in the first frame become occluded in the second. Such general-purpose detection of occlusion regions is dif-ficult and important because one-to-one correspondence of imaged scene points is needed for many tracking, video seg-mentation, and reconstruction algorithms. Our hypothesis is that an effective trained occlusion detector can be gener-ated on the basis of i) a broad spectrum of visual features, and ii) representative but synthetic training sequences. By using a Random Forest based framework for feature selec-tion and training, we found that the proposed feature set was sufficient to frequently assign a high probability of oc-clusion to just the pixels that were indeed becoming oc-cluded. Our extensive experiments on many sequences sup-port this finding, and while accuracy is certainly still scene-dependent, the proposed classifier could be a useful pre-processing step to exploit temporal information in video. 1