Focused and Deep Web Crawling-A Review

Abstract — In the exponentially growing Internet, web search engine companies needed to achieve scalability with large amount of hardware and network resources. For that they gave birth to a technique called focused web crawling to discover topic related information that can be used in online search. Focused crawlers dynamically browse the web by selecting the most favorable links to get maximum number of relevant pages with efficient utilization of bandwidth and time. Deep web is a vast repository in a web that are not always indexed by automated search engines. In this paper we are surveying the available techniques used for focused crawling and deep web crawling