Enforcing Temporal Consistency in Real-time Stereo Estimation

Abstract. Real-time stereo matching has many important applications in areas such as robotic navigation and immersive teleconferencing. When processing stereo sequences most existing real-time stereo algorithms calculate disparity maps for different frames independently without considering temporal consistency between adjacent frames. While it is known that temporal consistency information can help to produce better results, there is no efficient way to enforce temporal consistency in real-time applications. In this paper the temporal correspondences between disparity maps of adjacent frames are modeled using a new concept called disparity flow. A disparity flow map for a given view depicts the 3D motion in the scene that is observed from this view. An algorithm is developed to compute both disparity maps and disparity flow maps in an integrated process. The disparity flow map generated for the current frame is used to predict the disparity map for the next frame and hence, the temporal consistency between the two frames is enforced. All computations are performed in the image space of the given view, leading to an efficient implementation. In addition, most calculations are executed on programmable graphics hardware which further accelerates the processing speed. The current implementation can achieve 89 million disparity estimations per second on an ATI Radeon X800 graphic card. Experimental results on two stereo sequences demonstrate the effectiveness of the algorithm.