AUTOMATIC GENERATION OF A LARGE SCALE SEMANTIC SEARCH EVALUATION DATA-SET

compare the performance of information retrieval tech-niques in various settings, the data-sets which model these settings need to be generated. Although there are already available collections, such as those used in TREC conference series, which are used for evalua-tion of various retrieval tasks, there is a lack of collections which are specially developed for evaluation of the effectiveness of semantically en-hanced text retrieval techniques. In this paper, we propose an approach for the automatic generation of such data-sets, by using search engines query logs and data from human-edited web directories. The evaluation is performed by comparing the performance of Lucene, a popular syn-tactic search engine, and Concept Search, a search engine which extends Lucene’s syntactic search with semantics.