Experiments

This task was meant to compare the results of two different retrieval techniques: the first one was based on the words found in documents and query texts; the second one was based on the senses (concepts) obtained by disambiguating the words in documents and queries. The underlying goal was to come up with a more precise knowledge about the possible improvements brought by word sense disambiguation (WSD) in the information retrieval process. The proposed task structure was interesting in that it drew up a clear separation between the actors (humans or computers): those who provide the corpus, those who disambiguate it, and those who query it. Thus it was possible to test the universality and the interoperability of the methods and algorithms involved. Training and Testing Data The document corpus was created by merging two collections of English documents: LA Times 94 and Glasgow Herald 95 (166'000 documents with 470'000 unique words and 55 million word occurrences). This corpus was processed with two different word sense disambiguation algorithms: UBC [ubc07] and NUS [nus07], resulting in two different sets. The disambiguation process replaced each occurrence of a term (composed by one or more words) by an XML element containing the term identifier, an extracted lemma, a part-of-speech (POS) tag (noun, verb, adjective…), the original word form (WF) and a list of senses together with their respective scores. The senses were represented by WordNet 1.6 synset identifiers. For instance, the word "discovery " could be replaced by: <TERM ID="C041-19 " LEMA="discovery " POS="NN"> <WF>discovery</WF&gt