Estimation of three-dimensional motion and structure from images by using a temporally oriented approach

This dissertation explores the problem of estimating the 3-D motion and the structure of an object from video images using a new approach which looks for the temporal information prior to the spatial information. Since motion is observed over an extended period of time, we can reduce the number of features required by the conventional approach and improve significantly the estimation performance. In recovering the motion of a single particle or a rigid body, we prove that, under some conditions, the solution is unique. Regression relations between the unknown motion parameters and the projective trajectories are obtained for general particle motion and constant rigid motion. The method of maximum likelihood is used to estimate the motion. Using the nonlinear state estimation formulation, the extended Kalman filter is applied to obtain the estimate recursively. We propose an approach to estimate a general non-constant rigid motion in which the orders of translation and rotation can be arbitrary. We also show that for some special angular velocities, non-constant rigid motion has closed-form evolution, and discuss how to reduce the number of unknowns for a planar surface. We have addressed the problem of model mismatches for parameter jumping, undermodeling and overmodeling. We find that the model error makes the conventional approach break down. In order to solve this problem, we develop a new filter called Finite Lifetime Alternately Triggered Multiple Model Filter FLAT MMF). FLAT MMF is a filter composed of a number of identical conventional state estimation filters, each triggered alternately. After the last filter is triggered, the oldest one is triggered again and, so on. Experiments on the simulated trajectory and the real images show that FLAT MMF is quite effective in suppressing the model errors. For the particle motion without a depth change, we obtain the analytic, closed-form estimate for cases of model match and mismatch. We show that the filter that provides the best estimates dominates the final estimate. Finally, we show the potential of FLAT MMF for real-time object tracking