Reliability Verification of Search Engines ’ Hit Counts

Abstract. In this paper, we investigate the trustworthiness of search engines’ hit counts, numbers returned as search result counts. Since many studies adopt search engines ’ hit counts to estimate the popularity of input queries, the reliability of hit counts is indispensable for archiving trustworthy studies. However, hit counts are unreliable because they change, when a user clicks the “Search ” button more than once or clicks the “Next ” button on the search results page, or when a user queries the same term on separate days. In this paper, we analyze the characteristics of hit count transition by gathering various types of hit counts over two months by using 10,000 queries. The results of our study show that the hit counts with the largest search offset just before search engines adjust their hit counts are the most reliable. Moreover, hit counts are the most reliable when they are consistent over approximately a week