THE CONCEPTION OF INTEGRATING MUTITHREDED CRAWLER WITH PAGE RANK TECHNIQUE:A SURVEY

Web Crawler also well-known as “Web Robot”, “Web Spider ” or merely “Bot ” is software for downloading pages from the Web by design. Contrasting what the name may propose, a Web crawler does not in reality stir around computers connected to the Internet –as viruses or intelligent agents do – but only sends requests for documents on Web servers. The input to this software is starting or seed page. As the volume of the World Wide Web (WWW) grows, it became essential to parallelize a web crawling process, with the intention of finish downloading pages in a rational amount of time. Web crawler which employs multi-processing to permit multiple crawler processes running in concurrent manner. There are a lot of programs out there for web crawling but it required a WebCrawler that allowed trouble-free customization. In this paper we have discussed on crawling technique and how Page Rank can increase the efficiency of web crawling