Automatic image annotation through multi-topic text categorization

We propose a new framework for automatic image annotation through multi-topic text categorization. Given a test image, it is first converted into a text document using a visual codebook learnt from a collection of training images. Latent semantic analysis is then performed on the tokenized document to extract a feature vector based on a visual lexicon with its vocabulary items defined as either a codeword or a co-occurrence of multiple codewords. The high-dimension feature vector is finally compared with a set of topic models, one for each concept to be annotated, to decide on the top concepts related to the test image. These topic classifiers are discriminatively trained from images with multiple associations, including spatial, syntactic, or semantic relationship, between images and concepts. The proposed approach was evaluated on a Corel dataset with 374 keywords, and the TRECVID 2003 dataset with ten selected concepts. When compared with state-of-the-art algorithms for automatic image annotation on the Corel test set our system obtained the best results, although we only use a simple linear classification model based on just texture and color features. 1