Learning probabilistic models of the web

In the World Wide Web, myriads of hyperlinks connect doc-uments and pages to create an unprecedented, highly com-plex graph structure- the Web graph. This paper presents a novel approach to learning probabilistic models of the Web, which can be used to make reliable predictions about connec-tivity and information content of Web documents. The pro-posed method is a probabilistic dimension reduction tech-nique which recasts and unites Latent Semantic Analysis and Kleinberg's Hubs-and-Authorities algorithm in a statis-tical setting. This is meant to be a rst step towards the development of a statistical foundation for Web{related information tech-nologies. Although this paper does not focus on a partic-ular application, a variety of algorithms operating in the Web/Internet environment can take advantage of the pre-sented techniques, including search engines, Web crawlers, and information agent systems. 1 Introduction: Predictive Model