Semantic Scene Modeling and Retrieval

This book presents a novel image representation that allows to access natural scenes by local semantic description. During semantic modeling, local image regions are classified into semantic concepts classes such as water, rocks, and foliage. Images are represented through the frequency of occurrence of the local semantic concepts. This image representation is demonstrated to be well suited for modeling the semantic content of heterogeneous scene categories, and thus for categorization and retrieval. Furthermore, the image representation based on semantic modeling qualifies for ranking natural scenes according to their semantic similarity. This application is of special interest for content-based image retrieval systems that rely on the correct ordering of the returned images. In two psychophysical experiments, the human perception of the employed natural scenes has been studied. A categorization and a typicality ranking experiment showed that humans are very consistent in classifying scenes and in rating their semantic typicality with respect to five scene categories. Based on these findings, a novel perceptually plausible distance measure is introduced that allows to automatically rank natural scenes with a high correlation to the human ranking. Finally, the work discusses the problem of performance evaluation in content-based image retrieval systems. When searching for specific local semantic content, the retrieval results can be modeled statistically. Closed-form expressions for the prediction and the optimization of retrieval precision and recall are developed that permit to optimize precision and recall by up to 60