Live subtitling with speech recognition causes and consequences of text reduction

Speech technology has made it possible to use speech recognition for simultaneous subtitling of live television broadcasts via the technique of respeaking. Despite the considerable prior research into the quality of live subtitling using speech recognition, little research has focused on the quantitative aspects of subtitles. Although live subtitles are nearly always a reduced form of the spoken comments, the exact causes of text reduction are still largely unidentified. This study aims at a better understanding of the causes and consequences of text reduction in a live subtitling context. Three excerpts of an infotainment talk show were subtitled by twelve respeakers of the Flemish public television. They were instructed to do this in three different reduction conditions. Various subtitle features, such as reduction percentages and delay, as well as measures of the respeakers’ working memory were collected. Both a quantitative and qualitative analysis were carried out. In the quantitative analysis we opted for a multilevel analysis to take into account the hierarchical nature of the data. In the qualitative analysis, we discussed the effects of commonly used reduction strategies. The results show that reduction is not a random process. In contrast, it is largely determined by a number of external factors, viz. delay, amount of source text and the proportion of ‘full’ reductions. There is a large amount of evidence suggesting that respeakers prefer to omit certain comments rather than reducing them to a certain extent. It also appears that the decision to fully omit a comment seems not to be primarily based on the amount of input, while the decision to partially reduce is. Differences in the capacity of the working memory do not seem to affect text reduction as such. Finally, the qualitative analysis demonstrated that respeakers use a wide variety of strategies to reduce the spoken comments in order to limit the loss of information as much as possible.Real-time subtitling, Live subtitling, Respeaking, Voice-writing, Speech recognition, Keystroke logging, Reduction