Automatic Registration of LiDAR and Optical Imagery using Depth Map Stereo

Automatic fusion of aerial optical imagery and untex-tured LiDAR data has been of significant interest for gen-erating photo-realistic 3D urban models in recent years. However, unsupervised, robust registration still remains a challenge. This paper presents a new registration method that does not require priori knowledge such as GPS/INS in-formation. The proposed algorithm is based on feature cor-respondence between a LiDAR depth map and a depth map from an optical image. Each optical depth map is gener-ated from edge-preserving dense correspondence between the image and another optical image, followed by ground plane estimation and alignment for depth consistency. Our two-pass RANSAC with Maximum Likelihood estimation in-corporates 2D-2D and 2D-3D correspondences to yield ro-bust camera pose estimation. Experiments with a LiDAR-optical imagery dataset show promising results, without us-ing initial pose information. 1