The Accuracy and Value of Machine-Generated Image Tags Design and User Evaluation of an End-to-End Image Tagging System

Automated image tagging is a problem of great interest, due to the proliferation of photo sharing services. Researchers have achieved considerable advances in understanding mo-tivations and usage of tags, recognizing relevant tags from image content, and leveraging community input to recom-mend more tags. In this work we address several important issues in building an end-to-end image tagging application, including tagging vocabulary design, taxonomy-based tag refinement, classifier score calibration for effective tag rank-ing, and selection of valuable tags, rather than just accurate ones. We surveyed users to quantify tag utility and error tol-erance, and use this data in both calibrating scores from au-tomatic classifiers and in taxonomy based tag expansion. We also compute the relative importance among tags based on user input and statistics from Flickr. We present an end-to-end system evaluated on thousands of user-contributed pho-tos using 60 popular tags. We can issue four tags per image with over 80 % accuracy, up from 50 % baseline performance, and we confirm through a comparative user study that value-ranked tags are preferable to accuracy-ranked tags