Contour-based joint clustering of multiple segmentations

We present an unsupervised, shape-based method for joint clustering of multiple image segmentations. Given two or more closely-related images, such as nearby frames in a video sequence or images of the same scene taken under different lighting conditions, our method generates a joint segmentation of the images. We introduce a novel contour-based representation that allows us to cast the shape-based joint clustering problem as a quadratic semi-assignment problem. Our score function is additive. We use complex-valued affinities to assess the quality of matching the edge elements at the exterior bounding contour of clusters, while ignoring the contributions of elements that fall in the inte-rior of the clusters. We further combine this contour-based score with region information and use a linear program-ming relaxation to solve for the joint clusters. We evaluate our approach on the occlusion boundary data-set of Stein et al. 1