Journal Homepage: www.ijcst.org

Abstract – Content of the web page is the textual and graphical information that related to the topic of the page, which is the focus of web data mining and information retrieval. For web pages, the page content is the target of word-segmentation and indexing for search engine, corpus collection of news, reviews, blogs, etc. for knowledge management researches. Extracting content of the web pages correctly and efficiently improves the accuracy of following analysis for it significantly reduces the noise in the pages, and also alleviates the workload of indexing and segmentation. In this works, no uniform approach or model is presented to measure the importance of different nested portions in web pages. Through a user study, we found that people do have a consistent view about the importance of blocks in web pages. In this paper, we investigate how to find a model to automatically assign importance values to nested blocks in a web page. We define the block importance estimation as a learning problem. First, we use the VIPS (Vision-based Page Segmentation) algorithm to partition a web page and block in the webpage into semantic blocks with a hierarchy structure. Then spatial features (such as position, size) and content features (such as the number of images and links) are extracted to construct a feature vector for each block and nested blocks.. Based on analyzing the features of the pages, this approach could effectively extract contents from web pages. Experiments show good results comparing to related works