Surface measurement and tracking of human body parts from multi-image video sequences

This paper describes a method to measure and track moving surfaces of human body parts from multi-image video sequences acquired simultaneously by several cameras. The gained 3-D data can be of two different types: surface measurement of the visible parts of the human body at each time step of the sequence and surface tracking in the form of a vector field of 3-D trajectories (position, velocity and acceleration). The surface measurement process, which is based on multi-image photogrammetry, consists of five steps: calibration of the camera system, simultaneous acquisition of images from different views, establishment of corresponding points in the images, computation of their 3-D coordinates and, eventually, generation of a surface model. The high level of automation achieved in all the steps makes the processing of long image sequences possible. The tracking process is based on least squares matching techniques. The main idea is to track corresponding points in the multi-images through the sequence and compute their 3-D trajectories. When applied to all the points matched on the body, it results in a vector field of trajectories. Some key-points can be defined and tracked in the vector field, producing general 3-D information about the recorded movement. The main advantages of the presented method are: the capability to dynamically measure surface parts with high accuracy and the possibility to extract motion information from the acquired data without using markers. Two applications are presented to demonstrate the functionality of the proposed method: human face modelling and full body motion capture