How efficient are they in fact? : An evaluation of three web-based search engines

The purpose of this masters thesis is to evaluate the retrieval effectiveness of three different search engines on the Internet: Google, Yahoo! and MSN Search. We have constructed 25 topics and queries with varying contents, in order to test the width of the search engines as much as possible. In every search we have evaluated the first 20 hits in each of the search engines. The measures we have used are precision and cumulated gain. Precision is based on a binary scale and cumulated gain is calculated by means of a non-binary relevance scale. The relevance judgement in this thesis is based on a four-point relevance scale, which reaches from 0-3. Our method with the two different measures gave us the opportunity to compare how varying the result can be, depending on whether we use a binary or a non-binary scale. Eventually, we analysed the results we got in different DCVs to decide where in the results page the relevant documents are to be found; early or further down the ranked list. There were no considerable differences between the three search engines but Google achieved the highest results for both precision and cumulated gain. Regarding precision Yahoo! got the lowest end result, but for cumulated gain MSN Search performed the worst.Uppsatsnivå: