Towards Semantic Image Annotation With Keyword Disambiguation Using Semantic And Visual Knowledge

This paper deals with the semantic enrichment of automatic annotations of images. Since it par-tially tackles the Semantic Gap Problem, seman-tic image annotation has received a large atten-tion in the recent years. Nevertheless, the results of existing image annotation approaches are still not sufficient. We propose an original approach combining a priori knowledge (in our case, the WordNet lexical resource) and visual knowledge to build sense-tagged keywords-based annotation. First, a graph-based approach assigns a bag-of-keywords to a query image. Then, we propose to adapt a word sense disambiguation algorithm named SSI (Structural Semantic Interconnections), initially dedicated to text. We make two adapta-tions. First the grammar used in the SSI is modified to reflect the preponderance of semantic relations in image databases. Then, visual knowledge, includ-ing co-occurrence statistics in the visual domain and visual cues, is integrated. At last, a method to evaluate our approach is proposed.