Real-time filtering for human pose estimationusing multiple Kinects

This Master’s thesis proposes a working approach to combining data from multiple Kinect depth sensors in order to create stable pose estimates of a human user in real-time.The multi-camera approach is shown to increase the interaction area in which the user can move around, it gives more accurate estimates when the user is turning and it reduces issues with user occlusion compared to single-camera setups. In this report we implement and compare two different filtering techniques, particle filters and Kalman filters. We alsodiscuss different approaches to fuse data from multiple depth sensors based on the quality of the observations from the different sensors along with techniques to improve estimates such as applying body constraints. Both filtering approaches can be run on a normal laptop inreal-time, i.e. 30 Hz. When requiring real-time performance, the computationally efficient Kalman filter performs better than the particle filter overall in terms of stable estimations and performance. The quality of the particle filter is highly dependent on the number of particles that can be used before the frame rate drops below 30 frames per second. The implemented system provides a stable, fast and cost-efficient setup for motion capture and pose estimations of human users. There are important applications in virtual reality andto some extent also 3D games and rendered films that could benefit from the approaches discussed in this report