Registration for augmented reality

Augmented Reality (AR) refers to the insertion of virtual objects into real scenes. A key issue in AR is the registration or correct alignment of the virtual world with the real one. In the context of computer vision, this amounts to localizing the camera in a world coordinate frame. Among the constraints imposed by AR application are the need for real-time performance, unique solutions, robustness to image noise and acceptable performance with limited data input. Our contribution is in two areas. First, we assume an uncalibrated camera and no knowledge of either the 3D world or of correspondences between world objects and their images. We develop a novel decomposition of the fundamental matrix for certain restricted motions which allows for direct, closed-form recovery of the intrinsic and motion parameters of the camera. We test this algorithm in simulation and in real world experiments. Next, we consider a more realistic situation for most AR applications, that of known calibration, 3D data and world to image correspondences. Our main contribution is in this area. We develop a set of novel linear algorithms for both point and line correspondences, which depend on linearization of a system of quadratic equations derived from the geometry of the camera projection model. In both cases, we show how to recover the real solution from the potentially larger solution space associated with the linearized system. While there are other linear algorithms for point correspondences, we are aware of no other line based algorithm equivalent to ours. Unlike the commonly used optimization approaches, our algorithms are insensitive to issues of initialization and local minima. However, they are nearly as robust in the presence of image noise as the best optimization techniques with good initialization and better than competing linear methods. To prove the validity of our approach, we present extensive real experiments and simulations comparing our algorithms to existing linear and optimization approaches. We also present results on error sensitivity and error propagation for our algorithms