Simultaneous color consistency and depth map estimation for radiometrically varying stereo images

In this paper, we propose a new method that infers accu-rate depth maps and color-consistent images between ra-diometrically varying stereo images, simultaneously. In general, stereo matching and performing color consis-tency between stereo images are a chicken-and-egg prob-lem. Color consistency enhances the performance of stereo matching, while accurate correspondences from stereo dis-parities improve color consistency between stereo images. We devise a new iterative framework in which these two pro-cesses can boost each other. For robust stereo matching, we utilize the mutual information-basedmethod combined with the SIFT descriptor from which we can estimate the joint pdf in log-chromaticity color space. From this joint pdf, we can estimate a linear relationship between the correspond-ing pixels in stereo images. Using this linear relationship and the estimated depth maps, we devise a stereo color his-togram equalization method to make color-consistent stereo images which conversely boost the disparity map estima-tion. Experimental results show that our method produces both accurate depth maps and color-consistent stereo im-ages even for stereo images with severe radiometric differ-ences. 1