Choosing linguistics over vision to describe images

In this paper, we address the problem of automatically generating human-like descriptions for unseen images, given a collection of images and their corresponding human-generated descriptions. Previous attempts for this task mostly rely on visual clues and corpus statis-tics, but do not take much advantage of the semantic in-formation inherent in the available image descriptions. Here, we present a generic method which benefits from all these three sources (i.e. visual clues, corpus statis-tics and available descriptions) simultaneously, and is capable of constructing novel descriptions. Our ap-proach works on syntactically and linguistically moti-vated phrases extracted from the human descriptions. Experimental evaluations demonstrate that our formu-lation mostly generates lucid and semantically correct descriptions, and significantly outperforms the previous methods on automatic evaluation metrics. One of the significant advantages of our approach is that we can generate multiple interesting descriptions for an image. Unlike any previous work, we also test the applicabil-ity of our method on a large dataset containing complex images with rich descriptions