The Role of Context for Object Detection and Semantic Segmentation in the Wild

In this paper we study the role of context in existing state-of-the-art detection and segmentation approaches. Towards this goal, we label every pixel of PASCAL VOC 2010 de-tection challenge with a semantic category. We believe this data will provide plenty of challenges to the community, as it contains 520 additional classes for semantic segmenta-tion and object detection. Our analysis shows that near-est neighbor based approaches perform poorly on semantic segmentation of contextual classes, showing the variability of PASCAL imagery. Furthermore, improvements of exist-ing contextual models for detection is rather modest. In order to push forward the performance in this difficult sce-nario, we propose a novel deformable part-based model, which exploits both local context around each candidate de-tection as well as global context at the level of the scene. We show that this contextual reasoning significantly helps in detecting objects at all scales. 1