Instance Segmentation of Indoor Scenes using a Coverage Loss

Abstract. A major limitation of existing models for semantic segmen-tation is the inability to identify individual instances of the same class: when labeling pixels with only semantic classes, a set of pixels with the same label could represent a single object or ten. In this work, we in-troduce a model to perform both semantic and instance segmentation simultaneously. We introduce a new higher-order loss function that di-rectly minimizes the coverage metric and evaluate a variety of region features, including those from a convolutional network. We apply our model to the NYU Depth V2 dataset, obtaining state of the art results