A Higher-level Visual Representation for Semantic Learning in Image Databases

International audienceWith the availability of massive amounts of digital images in personal and on-line collections, effective techniques for navigating, indexing and searching images become more crucial. In this thesis, we rely on the image visual content as the main source of information to represent images. Starting from the bag of visual words (BOW) representation, a higher-level visual representation is learned where each image is modeled as a mixture of visual topics depicted in the image and related to high-level topics. First, we enhance the BOW representation by characterizing the spatial-color constitution of an image with a mixture of n Gaussians in the feature space. This leads to propose a novel descriptor, the Edge Context, which plays a role as a complementary descriptor in addition to the SURF descriptor. Such enhancements incorporate different image content information. Second, we introduce a new probabilistic topic model, Multilayer Semantic Significance Analysis (MSSA) model, in order to study a semantic inference of the constructed visual words. Consequently, we generate the Semantically Significant Visual Words (SSVWs). Third, we strengthen the discrimination power of SSVWs by constructing Semantically Significant Visual Phrases (SSVPs) from frequently co-occurring SSVWs that are semantically coherent. We partially bridge the intra-class visual diversity of the images by re-indexing the SSVWs and the SSVPs based on their distributional clustering. This leads to generate a Semantically Significant Invariant Visual Glossary (SSVIG) representation. Finally, we propose a new spatial weighting scheme and a Multiclass Vote-Based Classifier (MVBC) based on the proposed SSVIG representation. The large-scale extensive experimental results show that the proposed higher-level visual representation outperforms the traditional part-based image representations in retrieval, classification and object recognition