Efficent lattice rescoring using recurrent neural network language models

Recurrent neural network language models (RNNLM) have become an increasingly popular choice for state-of-the-art speech recogni-tion systems due to their inherently strong generalization perfor-mance. As these models use a vector representation of complete history contexts, RNNLMs are normally used to rescore N-best lists. Motivated by their intrinsic characteristics, two novel lattice rescor-ing methods for RNNLMs are investigated in this paper. The first uses an n-gram style clustering of history contexts. The second ap-proach directly exploits the distance measure between hidden history vectors. Both methods produced 1-best performance comparable with a 10k-best rescoring baseline RNNLM system on a large vocab-ulary conversational telephone speech recognition task. Significant lattice size compression of over 70 % and consistent improvements after confusion network (CN) decoding were also obtained over the N-best rescoring approach. Index Terms: recurrent neural network, language model, speech recognitio