Clothing co-parsing by joint image segmentation and labeling

This paper aims at developing an integrated system of clothing co-parsing, in order to jointly parse a set of cloth-ing images (unsegmented but annotated with tags) into se-mantic configurations. We propose a data-driven frame-work consisting of two phases of inference. The first phase, referred as “image co-segmentation”, iterates to extrac-t consistent regions on images and jointly refines the re-gions over all images by employing the exemplar-SVM (E-SVM) technique [23]. In the second phase (i.e. “region co-labeling”), we construct a multi-image graphical model by taking the segmented regions as vertices, and incorporate several contexts of clothing configuration (e.g., item loca-tion and mutual interactions). The joint label assignmen-t can be solved using the efficient Graph Cuts algorithm. In addition to evaluate our framework on the Fashionista dataset [30], we construct a dataset called CCP consist-ing of 2098 high-resolution street fashion photos to demon-strate the performance of our system. We achieve 90.29% / 88.23 % segmentation accuracy and 65.52 % / 63.89% recognition rate on the Fashionista and the CCP dataset-s, respectively, which are superior compared with state-of-the-art methods. 1