Search Engines for academic use : An evaluation of Google and Argos with queries from an academic discipline

The purpose of this MSc thesis is to examine the retrieval effectiveness of two Web search engines with queries from the academic discipline of Classical Studies. The two search engines are chosen to represent two different types Google as a broad general search engine and Argos as a specialised search engine for the subject of Classical Studies. The search engines are compared for precision among the first twenty results returned for thirty queries. In order to avoid bias in the study, the queries are based on real users' information needs. Due to the subjective character of the concept of relevance, the study is performed with five different experiments with five different definitions of relevance. The formula for calculating the metrics measures precision with weights for ranking effectiveness. Wilcoxon's signed rank test is used to control if there are any significant differences between the results. Analysis shows that Google is the top service of the two, since it performs better than Argos in all of the five experiments. The signed rank test also shows that there are significant differences between all of the results. One possible reason for Argos' lower results is the many duplicate links that it returned. Other explanations discussed are Argos' limited number of search facilities in comparison with Google, and that the ranking algorithm Argos uses is not advanced enough for Information Retrieval in this context. Lars Jonsson vann Collijnpriset 2003 med ovanstående uppsats.Uppsatsnivå: