Piecewise quadratic reconstruction of non-rigid surfaces from monocular sequences

Abstract. In this paper we present a new method for the 3D recon-struction of highly deforming surfaces (for instance a flag waving in the wind) viewed by a single orthographic camera. We assume that the sur-face is described by a set of feature points which are tracked along an image sequence. Most non-rigid structure from motion algorithms as-sume a global deformation model where a rigid mean shape component accounts for most of the motion and the deformation modes are small deviations from it. However, in the case of strongly deforming objects, the deformations become more complex and a global model will often fail to explain the intricate deformations which are no longer small lin-ear deviations from a strong mean component. Our proposed algorithm divides the surface into overlapping patches, reconstructs each of these patches individually using a quadratic deformation model and finally reg-isters them imposing the constraint that points shared by patches must correspond to the same 3D points in space. We show good results on challenging motion capture and real video sequences with strong defor-mations where global methods fail to achieve good reconstructions.