ResIn: A Combination of Results Caching and Index Pruning for High-performance Web Search Engines

Results caching is an efficient technique for reducing the query processing load, hence it is commonly used in real search engines. This technique, however, bounds the maxi-mum hit rate due to the large fraction of singleton queries, which is an important limitation. In this paper we propose ResIn- an architecture that uses a combination of results caching and index pruning to overcome this limitation. We argue that results caching is an inexpensive and effi-cient way to reduce the query processing load and show that it is cheaper to implement compared to a pruned index. At the same time, we show that index pruning performance is fundamentally affected by the changes in the query traffic that the results cache induces. We experiment with real query logs and a large document collection, and show that the combination of both techniques enables efficient reduc-tion of the query processing costs and thus is practical to use in Web search engines