Speaker-independent connected letter recognition with a multistate time delay neural network

The Multi-State Time Delay Neural Network (MS-TDNN) inte-grates a nonlinear time alignment procedure (DTW) and the high-accuracy phoneme spotting capabilities of a TDNN into a connec-tionist speech recognition system with word-level classification and error backpropagation. We present an MS-TDNN for recognizing continuously spelled letters, a task characterized by a small but highly confusable vocabulary. Our MS-TDNN achieves 98.5/92.0% word accuracy on speaker dependent/independent tasks, outper-forming previously reported results on the same databases. We pro-pose training techniques aimed at improving sentence level perfor-mance, including free alignment across word boundaries, word du-ration modeling and error backpropagation on the sentence rather than the word level. Architectures integrating submodules special-ized on a subset of speakers achieved further improvements.