3D Motion Reconstruction for Real-World Camera Motion

This paper addresses the problem of 3D motion recon-struction from a series of 2D projections under low recon-structibility. Reconstructibility defines the accuracy of a 3D reconstruction from 2D projections given a particular tra-jectory basis, 3D point trajectory, and 3D camera center trajectory. Reconstructibility accuracy is inherently related to the correlation between point and camera trajectories. Poor correlation leads to good reconstruction, high corre-lation leads to poor reconstruction. Unfortunately, in most real-world situations involving non-rigid objects (e.g. bod-ies), camera and point motions are highly correlated (i.e., slow and smooth) resulting in poor reconstructibility. In this paper, we propose a novel approach for 3D motion re-construction of non-rigid body motion in the presence of real-world camera motion. Specifically we: (i) propose the inclusion of a small number of keyframes in the video se-quence from which 3D coordinates are inferred/estimated to circumvent ambiguities between point and camera mo-tion, and (ii) employ a L1 penalty term to enforce a spar-sity constraint on the trajectory basis coefficients so as to ensure our reconstructions are consistent with the natural compressibility of human motion. We demonstrate impres-sive 3D motion reconstruction for 2D projection sequences with hitherto low reconstructibility. 1