Applying pattern mining to web information extraction

Abstract. Information extraction (IE) from semi-structured Web doc-uments is a critical issue for information integration systems on the In-ternet. Previous work in wrapper induction aim to solve this problem by applying machine learning to automatically generate extractors. For example, WIEN, Stalker, Softmealy, etc. However, this approach still re-quires human intervention to provide training examples. In this paper, we propose a novel idea to IE, by repeated pattern mining and multiple pat-tern alignment. The discovery of repeated patterns are realized through a data structure call PAT tree. In addition, incomplete patterns are further revised by pattern alignment to comprehend all pattern instances. This new track to IE involves no human eort and content-dependent heuris-tics. Experimental results show that the constructed extraction rules can achieves 97 percent extraction over fourteen popular search engines