Improved Skeleton Tracking by Duplex Kinects: A Practical Approach for Real-Time Applications

Recent development of per-frame motion extraction method can generate the skeleton of human motion in real-time with the help of RGB-D cameras such as Kinect. This leads to an economic device to provide human motion as input for real-time applications. As generated by a single-view im-age plus depth information, the extracted skeleton usually has problems of unwanted vibration, bone-length variation, self-occlusion, etc. This paper presents an approach to over-come these problems by synthesizing the skeletons gener-ated by duplex Kinects, which capture the human motion in different views. The major technical difficulty of this syn-thesis comes from the inconsistency of two skeletons. Our algorithm is formulated under the constrained optimization framework by using the bone-lengths as hard constraints and the tradeoff between inconsistent joint positions as soft con-straints. Schemes are developed to detect and re-position the problematic joints generated by per-frame method from du-plex Kinects. As a result, we develop an easy, cheap and fast approach that can improve the skeleton of human motion at an average speed of 5ms per frame