Hierarchical semantic segmentation of image scene with object labeling

Abstract Semantic segmentation of an image scene provides semantic information of image regions while less information of objects. In this paper, we propose a method of hierarchical semantic segmentation, including scene level and object level, which aims at labeling both scene regions and objects in an image. In the scene level, we use a feature-based MRF model to recognize the scene categories. The raw probability for each category is predicted via a one-vs-all classification mode. The features and raw probability of superpixels are embedded into the MRF model. With the graph-cut inference, we get the raw scene-level labeling result. In the object level, we use a constraint-based geodesic propagation to get object segmentation. The category and appearance features are utilized as the prior constraints to guide the direction of object label propagation. In this hierarchical model, the scene-level labeling and the object-level labeling have a mutual relationship, which regions and objects are optimized interactively. The experimental results on two datasets show the well performance of our method