FOREST: Focused Object Retrieval by Exploiting Significant Tag Paths

Content-intensive websites, e.g., of blogs or news, present pages that contain Web articles automatically generated by content management systems. Iden-tification and extraction of their main content is critical in many applications, such as indexing or classification. We present in this paper a novel unsuper-vised approach for the extraction of Web articles from dynamically-generated Web pages. Due to the template sophistication and the presence of various dynamic and topically-different contents in the page, standard unsupervised wrapper induction approaches are not sufficient on their own to identify the content of interest. State-of-the art methods for main content extraction that operate at single page level fail to leverage the common template and functionality of Web pages generated by the same Web source. Our system, called Forest, combines structural and information-based features to precisely target the main content generated by a Web source, and published in associated Web pages. By inferring a wrapper that characterizes the content generation patterns of the source, we harness the template inertia to create an efficient technique for Web article extraction. We extensively evaluate the Forest algorithm with respect to various baselines and Web source datasets, and report improved results over state-of-the art techniques in content extraction.