A conceptual framework for indexing visual information at multiple levels

In this paper, we present a conceptual framework for indexing different aspects of visual information. Our framework unifies concepts from the literature in diverse fields such as cognitive psychology, library sciences, art, and the more recent contentbased retrieval. We present multiple level structures for visual and non-visual information. The ten-level visual structure presented provides a systematic way of indexing images based on syntax (e.g., color, texture, etc.) and semantics (e.g., objects, events, etc.), and includes distinctions between general concept and visual concept. We define different types of relations (e.g., syntactic, semantic) at different levels of the visual structure, and also use a semantic information table to summarize important aspects related to an image. While the focus is on the development of a conceptual indexing structure, our aim is also to bring together the knowledge from various fields, unifying the issues that should be considered when building a digital image library. Our analysis stresses the limitations of state of the art content-based retrieval systems and suggests areas in which improvements are necessary. Keywords: image indexing, image classification, conceptual indexing, content-based retrieval, MPEG-7. 1