Automatic discovery of groups of objects for scene understanding

Objects in scenes interact with each other in complex ways. A key observation is that these interactions man-ifest themselves as predictable visual patterns in the im-age. Discovering and detecting these structured patterns is an important step towards deeper scene understanding. It goes beyond using either individual objects or the scene as a whole as the semantic unit. In this work, we promote “groups of objects”. They are high-order composites of ob-jects that demonstrate consistent spatial, scale, and view-point interactions with each other. These groups of objects are likely to correspond to a specific layout of the scene. They can thus provide cues for the scene category and can also prime the likely locations of other objects in the scene. It is not feasible to manually generate a list of all pos-sible groupings of objects we find in our visual world. Hence, we propose an algorithm that automatically dis-covers groups of arbitrary numbers of participating ob-jects from a collection of images labeled with object cat-egories. Our approach builds a 4-dimensional transform space of location, scale and viewpoint, and efficiently iden-tifies all recurring compositions of objects across images. We then model the discovered groups of objects using the deformable parts-based model. Our experiments on a vari-ety of datasets show that using groups of objects can signif-icantly boost the performance of object detection and scene categorization. 1