Automatic synchronization of electronic and audio books via TTS alignment and silence filtering

The e-book industry is starting to flourish due, in part, to the availability of affordable and user-friendly e-book read-ers. As users are increasingly moving from traditional paper books to e-books, there is an opportunity to reinvent and en-hance their reading experience, for example, by leveraging the multimedia capabilities of these devices in order to turn the act of reading into a real multimedia experience. In this paper, we focus on the augmentation of the written text with its associated audiobook, so that users can listen to the book they are (currently) reading. We propose an audiobook-to-ebook alignment system by applying a Text-to-Speech (TTS)-based text to audio alignment algorithm, and enhance it with a silence filtering algorithm to cope with the difference on reading style between the TTS output and the speakers in the ebook environment. Experiments done using 12 five-minute excerpts of 6 different audio-books (read by men and women) yield usable word alignment errors below 120ms for 90 % of the words. Finally, we also show a user interface implemen-tation in the Ipad for synchronized e-book reading while lis-tening to the associated audiobook. Index Terms — e-book, audiobook, multimodal synchro-nization, audio processing, TTS alignmen