Web Proxy Server with Intelligent Prefetcher for Dynamic Pages Using Association Rules

The growth of the World Wide Web has emphasized the need for improved user latency. Primarily, two techniques, i.e. caching and prefetching are being used for improving user latency. Several studies have been conducted on cache replacement policies for improving cache hit ratio. On prefetching side, studies have been conducted on prefetching models based on decision tree, Markov chain, and path analysis. Increasing use of dynamic pages, frequent changes in the site structure and user access patterns on the internet have limited the efficacy of caching techniques and emphasized the need for prefetching with a predictive model which is easier to build and update and have improved predictive performance. In this project, we study the existing caching and prefetching techniques and explore as to how the knowledge of user access patterns discovered through Web Mining and the information on site structure can be used to predict future requests by an user. In a web site certain sets of pages exhibit strong correlation with each other, which manifest in user access patterns. Such set correlations can be discovered in the form association rules by Web Mining of server logs. We accordingly propose a prefetching model based association rules. We also present a predictive model based on site structure where certain number of children of the current page are prefetched. We also present a design and prototype implementation of a proxy server, which use these models for prefetching and conduct experiments to evaluate the performance of these two models. In our experiments we find that the association rules prefetching model has better predictive value than the site structure model and gives good cache hit ratio without much additional traffic load