Repurposing Corpora for Speech Repair Detection: Two Experiments

Unrehearsed spoken language often contains many disfluencies. If we want to correctly interpret the content of spoken language, we need to be able to detect these disfluencies and deal with them appropriately. In the work de-scribed here, we use a statistical noisy channel model to detect disfluencies in transcripts of spoken language. Like all statistical approaches, this is natu-rally very data-hungry; however, cor-pora containing transcripts of unre-hearsed spoken language with disflu-encies annotated are a scarce resource, which makes training difficult. We address this issue in the follow-ing ways: First, since written textual corpora are much more abundant than speech corpora, we see whether using a large text corpus to increase the data available to our language model com-ponent delivers an improvement. Sec-ond, given that most spoken language corpora are not annotated with disflu-encies, we explore the use of Expecta-tion Maximisation to mark the disflu-encies in such corpora, so as to increase the data availability for our complete model. In neither case do we see an improve-ment in our results. We discuss these results and the possible reasons for the negative outcome.