Simultaneous pose and non-rigid shape with particle dynamics

In this paper, we propose a sequential solution to simul-taneously estimate camera pose and non-rigid 3D shape from a monocular video. In contrast to most existing ap-proaches that rely on global representations of the shape, we model the object at a local level, as an ensemble of par-ticles, each ruled by the linear equation of the Newton’s sec-ond law of motion. This dynamic model is incorporated into a bundle adjustment framework, in combination with simple regularization components that ensure temporal and spatial consistency of the estimated shape and camera poses. The resulting approach is both efficient and robust to several ar-tifacts such as noisy and missing data or sudden camera motions, while it does not require any training data at all. Validation is done in a variety of real video sequences, in-cluding articulated and non-rigid motion, both for continu-ous and discontinuous shapes. Our system is shown to per-form comparable to competing batch, computationally ex-pensive, methods and shows remarkable improvement with respect to the sequential ones. 1