TextTiling: A Quantitative Approach to Discourse Segmentation

This paper presents TextTiling, a method for partitioning full-length text documents into coherent multiparagraph units. The layout of text tiles is meant to reflect the pattern of subtopics contained in an expository text. The approach uses lexical analyses based on tf.idf, an information retrieval measurement, to determine the extent of the tiles, incorporating thesaural information via a statistical disambiguation algorithm. The tiles have been found to correspond well to human judgements of the major subtopic boundaries of science magazine articles