Computing Similarity Transformations from Only Image Correspondences

We propose a novel solution for computing the relative pose between two generalized cameras that includes recon-ciling the internal scale of the generalized cameras. This approach can be used to compute a similarity transforma-tion between two coordinate systems, making it useful for loop closure in visual odometry and registering multiple structure from motion reconstructions together. In contrast to alternative similarity transformation methods, our ap-proach uses 2D-2D image correspondences thus is not sub-ject to the depth uncertainty that often arises with 3D points. We utilize a known vertical direction (which may be easily obtained from IMU data or vertical vanishing point detec-tion) of the generalized cameras to solve the generalized relative pose and scale problem as an efficient Quadratic Eigenvalue Problem. To our knowledge, this is the first method for computing similarity transformations that does not require any 3D information. Our experiments on syn-thetic and real data demonstrate that this leads to improved performance compared to methods that use 3D-3D or 2D-3D correspondences, especially as the depth of the scene increases. 1