Threading Fundamental Matrices

this paper, we wish to find a consistent set of camera matrices in a direct manner, i.e., without reconstructing the 3D scene as an intermediate step. To this end, we introduce a new result on the connection between fundamental matrices and the trifocal tensor on a given triplet of images. Applying this result on a sliding window of triplets of images provides an efficient method for concatenating camera matrices along an extended sequence without resorting to 3D structure. The connection is based on a representation of the tensor as a function of the elements of two consecutive fundamental matrices and a homography matrix of some arbitrary reference plane. An interesting property of this representation is that each camera matrix is guaranteed to be consistent with the previously recovered camera matrices. That is, all camera matrices will be recovered due to the same reference plane. By repeated application of the basic result, we call a threading operation, on a sliding window of triplets of views, we obtain a consistent sequence of camera matrices (and the fundamental matrix and trifocal tensors as well). Since the threading operation is linear, we have an on-line structure-frommotion algorithm that has only a single nonlinear constraintthe rank-2 constraint of the first fundamental matrix. The rest of the camera matrices are recovered linearly one by one. Potential use of the threading operation includes ego-motion estimation or image stabilizatio