Complementarity and Similarity: Relationships Between Text-Mined Terms and Social Tags for Image Description

In this paper, we present our results on comparing the language of social tags with text-mined terms for images. We have developed a novel modification of the standard term frequency/inverse document frequency metric (tf*idf) (Salton & Buckley 1988) over tags and terms to identify and filter terms which discriminate images for searchers. Since tags serve as additional input, we refer to this modification as the T-tf*idf Measure, i.e. Tags-term frequency as an inverse of document frequency, where "document" in this case refers to the either the tag or term dataset. We present the results of several variations on this measure, and demonstrate the impact on output. We discuss evaluation of our results on the ability of the metric to reflect human judgments through experiments which illustrate the value of the approach