Investigation on language modelling approaches for open vocabulary speech recognition

By definition, words that are not present in a recognition vocabulary are called out-of-vocabulary (OOV) words. Recognition of unseen or new words is an important feature that is always desired in any real-world large vocabulary continuous speech recognition (LVCSR) system. However, human languages are complex in nature due to wide varieties of morphological richness such as inflections, derivations and compounding. For instance, language models for morphologically rich languages like German, Polish, Slovene, etc, often have high OOV rates, data sparsity and rather poor generalization of unseen sequences. In spite of the substantial amount of work that has been carried out to recognize unseen words in recent decades, many issues related to open vocabulary problem still exist, especially, under large vocabulary conditions. This dissertation addresses some of the core issues and makes an attempt to solve them by investigating and introducing different types of hybrid and hierarchical language models, supported by detailed experimental analysis. Careful selection of sub-word unit is necessary in a hybrid language model, as it has a large impact on OOV rate, data sparsity and recognition issues. Different types of sub-word unit, such as morphemes, syllables and graphones are investigated on selected morphologically rich languages. The traditional hybrid approach uses only sub-words, which is not robust in-terms of reducing word error rates on large vocabulary tasks. This work investigates different types of count-based hybrid language models. One method is to use an optimal number of full words and sub-words. Further extensions include the use of an optimal number of full words, sub-words and sub-word graphones based on word frequencies. The advantage of using two or three different types of units in a hybrid language model is that it helps improve recognition of OOVs and also compensates for weaker contexts, and reduces data sparsity to some extent. In addition, this work also investigates maximum entropy and long short-term memory network hybrid language models. A maximum entropy approach is combined within class-based language modelling framework. Additionally, novel extensions are proposed in the hierarchical language modelling approach, where a full word language model and a character level language model are directly used during decoding in a hierarchical manner to recognize in-vocabulary and OOV words, respectively, for LVCSR tasks. Sequence normalization using a prefix tree approach is applied to hierarchical language models. Variants of the hierarchical approach are introduced by incorporating weighted and non-weighted character language models, multi class character language models, and grapheme to phoneme models. These types of language model guarantee zero OOV rate. Alternatively, a properly normalized combined interpolated language model is introduced that also uses a full word language model and a character level language model during decoding, exploiting within word context or across word context at a character level for OOV recognition