Incorporating Scene Context and Object Layout into Appearance Modeling

A scene category imposes tight distributions over the kind of objects that might appear in the scene, the appear-ance of those objects and their layout. In this paper, we propose a method to learn scene structures that can encode three main interlacing components of a scene: the scene category, the context-specific appearance of objects, and their layout. Our experimental evaluations show that our learned scene structures outperform state-of-the-art method of Deformable Part Models in detecting objects in a scene. Our scene structure provides a level of scene understanding that is amenable to deep visual inferences. The scene struc-tures can also generate features that can later be used for scene categorization. Using these features, we also show promising results on scene categorization. 1