A hierarchical conditional random field model for labeling and segmenting images of street scenes

Simultaneously segmenting and labeling images is a fun-damental problem in Computer Vision. In this paper, we introduce a hierarchical CRF model to deal with the prob-lem of labeling images of street scenes by several distinc-tive object classes. In addition to learning a CRF model from all the labeled images, we group images into clusters of similar images and learn a CRF model from each cluster separately. When labeling a new image, we pick the closest cluster and use the associated CRF model to label this im-age. Experimental results show that this hierarchical image labeling method is comparable to, and in many cases supe-rior to, previous methods on benchmark data sets. In addi-tion to segmentation and labeling results, we also showed how to apply the image labeling result to rerank Google similar images. 1