General approach for determining 3D motion and structure of multiple objects from image trajectories

We present a general approach to determine 3D motion and structure of multiple objects undergoing arbitrary motions. We segment the scene based on 3D motion parameters. First, the general motion model is fitted to each single trajectory. For this nonlinear fitting, initial estimates are obtained by a linear multiple motion SFM algorithm using the first two frames. Next, trajectories are clustered into groups corresponding to different moving objects. In our approach, discontinuous trajectories, resulting from occlusion, are also allowed. Finally, the multiple trajectory fitting is applied to each trajectory group to improve the estimates further. Our simulation results show that the proposed method is robust