Joint Segmentation and Pose Tracking of Human in Natural Videos∗

We propose an on-line algorithm to extract a human by foreground/background segmentation and estimate pose of the human from the videos captured by moving cameras. We claim that a virtuous cycle can be created by appropri-ate interactions between the two modules to solve individ-ual problems. This joint estimation problem is divided into two subproblems, foreground/background segmentation and pose tracking, which alternate iteratively for optimization; segmentation step generates foreground mask for human pose tracking, and human pose tracking step provides fore-ground response map for segmentation. The final solution is obtained when the iterative procedure converges. We eval-uate our algorithm quantitatively and qualitatively in real videos involving various challenges, and present its out-standing performance compared to the state-of-the-art tech-niques for segmentation and pose estimation. 1