Dense Depth Map Reconstruction from Sparse Measurements Using a Multilayer Conditional Random Field Model

Abstract—Acquiring accurate dense depth maps is crucial for accurate 3D reconstruction. Current high quality depth sensors capable of generating dense depth maps are expensive and bulky, while compact low-cost sensors can only reliably generate sparse depth measurements. We propose a novel multilayer conditional random field (MCRF) approach to reconstruct a dense depth map of a target scene given the sparse depth measurements and corresponding photographic measurements obtained from stereophotogrammetric systems. Estimating the dense depth map is formulated as a maximum a posteriori (MAP) inference problem where a smoothness prior is assumed. Our MCRF model uses the sparse depth measurement as an additional observation layer and describes relations between nodes with multivariate feature functions based on the depth and photographic measurements. The method is first qualitatively analyzed when performed on data collected with a compact stereo camera, then quantitative performance is measured using the Middlebury stereo vision data for ground truth. Experimental results show our method performs well for reconstructing simple scenes and has lower mean squared error compared to other dense depth map reconstruction methods. Keywords-Dense depth map; Multilayer conditional random field; Stereophotogrammetry; 3D visualisation I