Selective Integration: A Model for Disparity Estimation

Local disparity information is often sparse and noisy, which creates two conflicting demands when estimating disparity in an image re-gion: the need to spatially average to get an accurate estimate, and the problem of not averaging over discontinuities. We have devel-oped a network model of disparity estimation based on disparity-selective neurons, such as those found in the early stages of process-ing in visual cortex. The model can accurately estimate multiple disparities in a region, which may be caused by transparency or oc-clusion, in real images and random-dot stereograms. The use of a selection mechanism to selectively integrate reliable local disparity estimates results in superior performance compared to standard back-propagation and cross-correlation approaches. In addition, the representations learned with this selection mechanism are con-sistent with recent neurophysiological results of von der Heydt, Zhou, Friedman, and Poggio [8] for cells in cortical visual area V2. Combining multi-scale biologically-plausible image processing with the power of the mixture-of-experts learning algorithm represents a promising approach that yields both high performance and new insights into visual system function. Selective Integration: A Model for Disparity Estimation 867