JORDT, KOCH: FAST TRACKING OF DEFORMABLE OBJECTS 1 Fast Tracking of Deformable Objects in Depth and Colour Video

One challenge in computer vision is the joint reconstruction of deforming objects from colour and depth videos. So far, a lot of research has focused on deformation re-construction based on colour images only, but as range cameras like the recently released Kinect become more and more common, the incorporation of depth information becomes feasible. In this article a new method is introduced to track object deformation in depth and colour image data. A NURBS based deformation function allows to decouple the geometrical object complexity from the complexity of the deformation itself, providing a low dimen-sional space to describe arbitrary ’realistic ’ deformations. While modelling the tracking objective as an analysis by synthesis problem, which is robust but usually computation-ally expensive, a set of optimisations is introduced, allowing a very fast calculation of the resulting error function. With a fast semi-global search a system is established that is capable of tracking complex deformations of large objects (6000 triangles and more) with more than 6Hz on a common desktop machine. The algorithm is evaluated using simulated and real data, showing the robustness and performance of the approach.