Int J Comput Vis DOI 10.1007/s11263-015-0804-2 3D Trajectory Reconstruction under Perspective Projection

Abstract We present an algorithm to reconstruct the 3D trajectory of a moving point from its correspondence in a collection of temporally non-coincidental 2D perspec-tive images, given the time of capture that produced each image and the relative camera poses at each time instant. Triangulation-based solutions do not apply, as multiple views of the point may not exist at each time instant. We represent a 3D trajectory using a linear combination of compact tra-jectory basis vectors, such as the discrete cosine transform basis, that have been shown to approximate object indepen-dence. We note that such basis vectors are also coordinate independent, which allows us to directly use camera poses estimated from stationary areas in the scene (in contrast to nonrigid structure from motion techniques where cameras are simultaneously estimated). This reduces the reconstruc-tion optimization to a linear least squares problem, allow-ing us to robustly handle missing data that often occur due to motion blur, texture deformation, and self occlusion. We present an algorithm to determine the number of trajectory basis vectors, individually for each trajectory via a cross val-idation scheme and refine the solution by minimizing the Communicated by Jun Sato