Clickage: Towards bridging semantic and intent gaps via mining click logs of search engines

The semantic gap between low-level visual features and high-level semantics has been investigated for decades but still remains a big challenge in multimedia. When “search ” be-came one of the most frequently used applications, “intent gap”, the gap between query expressions and users ’ search intents, emerged. Researchers have been focusing on three approaches to bridge the semantic and intent gaps: 1) de-veloping more representative features, 2) exploiting better learning approaches or statistical models to represent the semantics, and 3) collecting more training data with better quality. However, it remains a challenge to close the gaps. In this paper, we argue that the massive amount of click data from commercial search engines provides a data set that is unique in the bridging of the semantic and intent gap. Search engines generate millions of click data (a.k.a. image-query pairs), which provide almost “unlimited ” yet strong connections between semantics and images, as well as connections between users ’ intents and queries. To study the intrinsic properties of click data and to investigate how to effectively leverage this huge amount of data to bridge semantic and intent gap is a promising direction to advance multimedia research. In the past, the primary obstacle is that there is no such dataset available to the public research community. This changes as Microsoft has released a new large-scale real-world image click data to public. This pa-per presents preliminary studies on the power of large-scale click data with a variety of experiments, such as building large-scale concept detectors, tag processing, search, defini-tive tag detection, intent analysis, etc., with the goal to inspire deeper researches based on this dataset