COMBINING CONTENT AND CONTEXT INFORMATION FOR SEMANTIC IMAGE ANALYSIS AND CLASSIFICATION

In this paper, a learning approach to semantic image analysis and classification is proposed that combines global and local infor-mation, with explicitly defined knowledge in the form of an ontol-ogy. The ontology specifies the selected domain, its sub-domains, the concepts related to each sub-domain as well as contextual in-formation. Support Vector Machines (SVMs) are employed in order to provide image classification to one of the defined sub-domains based on global image descriptions and, after a segmentation al-gorithm is applied, to perform an initial mapping between region low-level visual features and the concepts in the ontology. Then, a decision function, that receives as input the region to concepts associations together with contextual information, realizes image classification based on local-level information. The contextual in-formation used is in the form of frequency of appearance of each concept in every particular sub-domain. A fusion mechanism com-bines the intermediate classification results, provided by the local-and global-level information processing, and decides on the final classification. A Genetic Algorithm (GA) is employed for optimiz-ing the fusion process. Experiments with images from the personal collection domain demonstrate the performance of the proposed ap-proach. 1