Detecting the Noise from Web Pages Using Entropy Measure

The rapid expansion of the Internet has madeWeb a popular place for disseminating andcollecting information from the web. The noisyitems in web pages are one of the majorproblems to extract the main contents. It is alsoimportant how to detect noises and distinguishvaluable information from noisy data within asingle Web page. In this paper, we propose anoise detection technique is based on theDocument Object Model (DOM) tree. In DOMtree, weight of each node calculated by tf-idfscheme is added in entropy measure to get therespective value, which will be compared with athreshold value. Those less than threshold valueare regarded as noise. Experimental results on arange of datasets using precision and recallmeasure show that our framework can improvenoise detection accuracy