News image annotation on a large parallel text-image corpus

In this paper, we present a multimodal parallel text-image corpus, and propose an image annotation method that exploit the textual infor-mation associated with images. Our corpus contains news articles composed of a text, images and image captions. In our experiments, we use the text of the articles as a textual information source to annotate images, and image captions as a groundtruth to evaluate our annotation algorithm. Our annotation method identifies named entities in the texts, and associates them with high-level visual concepts detected in the images (in this paper, faces and logos). Our experiments show that, although it is very simple, our annotation method achieves an acceptable accuracy on our real-world news corpus. 1