A Dynamic Programming Approach to Maximizing Tracks for Structure from Motion

{raphael.grasset, mark.billinghurst} @ hitlabnz.org Abstract. We present a novel algorithm for improving the accuracy of structure from motion on video sequences. Its goal is to efficiently re-cover scene structure and camera pose by using dynamic programming to maximize the lengths of putative keypoint tracks. By efficiently dis-carding poor correspondences while maintaining the largest possible set of inliers, it ultimately provides a robust and accurate scene reconstruc-tion. Traditional outlier detection strategies, such as RANSAC and its derivatives, cannot handle high dimensional problems such as structure from motion over long image sequences. We prove that, given an estimate of the camera pose at a given frame, the outlier detection is optimal and runs in low order polynomial time. The algorithm is applied on-line, pro-cessing each frame in sequential order. Results are presented on several indoor and outdoor video sequences processed both with and without the proposed optimization. The improvement in average reprojection errors demonstrates its effectiveness.