Towards Unified Human Parsing and Pose Estimation

We study the problem of human body configuration anal-ysis, more specifically, human parsing and human pose es-timation. These two tasks, i.e. identifying the semantic re-gions and body joints respectively over the human body image, are intrinsically highly correlated. However, pre-vious works generally solve these two problems separately or iteratively. In this work, we propose a unified frame-work for simultaneous human parsing and pose estima-tion based on semantic parts. By utilizing Parselets and Mixture of Joint-Group Templates as the representations for these semantic parts, we seamlessly formulate the hu-man parsing and pose estimation problem jointly within a unified framework via a tailored And-Or graph. A novel Grid Layout Feature is then designed to effectively cap-ture the spatial co-occurrence/occlusion information be-tween/within the Parselets and MJGTs. Thus the mutually complementary nature of these two tasks can be harnessed to boost the performance of each other. The resultant uni-fied model can be solved using the structure learning frame-work in a principled way. Comprehensive evaluations on two benchmark datasets for both tasks demonstrate the ef-fectiveness of the proposed framework when compared with the state-of-the-art methods. 1