Delta-TSR: a description of spatial relationships between objects for image retrieval

This article presents ∆-TSR, a new image content representation exploiting the spatial relationships existing between its objects of interest. This approach provides two types of descriptions: with ∆-TSR3D, images are represented by geometric relationships between triplets of objects using triangle angles, while ∆-TSR5D enriches ∆-TSR3D by exploiting the orientation of the objects. The approach is generalized to polygonal spatial relationships, from triplets to tuples of objects. All these descriptions are invariant to translation, 2D rotation, scale and flip. A semi-local representation of the relationships is also proposed, making the description robust to viewpoint changes, if required by the application. ∆-TSR can be applied not only to symbolic images (where objects are represented by labels or icons) but also to contents represented by low-level visual features such as interest points. In this work, we evaluate ∆-TSR for image and sub-image retrieval under the query-by-example paradigm, on contents represented with interest points in a bag-of- features representation. We show that ∆-TSR improves two state-of-the-art approaches, in terms of quality of retrieval as well as of execution time. The experiments also highlight its effectiveness and scalability against large image databases.ou