Indexing weblogs one post at a time

In order to perform analysis over weblogs, we must first iden-tify the appropriate unit of a weblog that corresponds to a document. We argue in the paper that, for weblogs, the cor-rect unit is the weblog post. A weblog post is a structured document with the following fields: date, timestamp, title, content, permalink and author. We present our approach for segmenting weblogs into posts, which breaks down into sev-eral steps: (1) automatic feed discovery; (2) feed-guided seg-mentation, using the weblog feed and HTML; and (3) model-based weblog segementation