Automatic extraction of information blocks using pat trees

Information extraction from semi-structured Web documents is a critical issue for software agents on the Internet. Previous work in wrapper induction aim to solve this problem by applying machine learning to automatically generate extrac-tors, but this approach still requires human inter-vention to provide training examples. In this paper, we present a novel approach that extracts informa-tion blocks without training examples using a data structure called a PAT tree. PAT trees allow the system to eÆciently recognize repeated patterns in a semi-structured Web page. From these repeated patterns, information blocks can be easily located based on some domain independent selection cri-teria. The entire system runs automatically with-out any human intervention. Experimental results show that our approach performs well with a re-call rate near 90 percent on a wide range of output pages of popular search engines.