Detecting Structural Metadata with Decision Trees and Transformation-Based Learning

The regular occurrence of disuencies is a distinguishing characteristic of spontaneous speech. Detecting and removing such disu-encies can substantially improve the usefulness of spontaneous speech transcripts. This pa-per presents a system that detects various types of disuencies and other structural information with cues obtained from lexical and prosodic information sources. Specically, combina-tions of decision trees and language models are used to predict sentence ends and interruption points and, given these events, transformation-based learning is used to detect edit disuen-cies and conversational llers. Results are re-ported on human and automatic transcripts of conversational telephone speech.