Automatic Image Annotation with Weakly Labeled Dataset∗

It is very attractive to exploit weakly-labeled image dataset for multi-label annotation applications. In our paper the meaning of the terminology weakly labeled is threefold: i) only a small subset of the available images are labeled; ii) even for the labeled image, the given labels may be uncor-rect or incomplete; iii) the given labels do not provide the exact object locations in the images. A novel method is developed to predict the multiple labels for images and to provide region-level labels for the objects. We cluster the im-age regions to learn several region-exemplars and predict the label vector for each image region as a locally weighted aver-age of the label vectors on exemplars. By investigating the label confidence matrix for the region-exemplars from dif-ferent perspectives (column picture and row picture), we sufficiently leverage the visual contexts, the semantic con-texts, and the consistency between similarities in the visual feature space and semantic label space. Experimental re-sults on real web images demonstrate the effectiveness of the proposed method