Discriminative training of linear transformations and mixture density splitting for speech recognition

Discriminative training has been established as an effective technique for training the acoustic model of an automatic speech recognition system. It reduces the word error rate as compared to standard maximum likelihood training. This thesis concerns itself with training reduced-rank linear transformations for reducing the number of parameters. Conversely, it is also investigated if a small robust model can be split into a larger model, for robust initialization. Previous work has shown the usefulness of discriminatively trained log-linear acoustic models. These have been shown to cover Gaussian single density and mixture models. Log-linear training i.e. convex optimization can also be used to train linear feature transformations. The main focus of this thesis is to explore log-linear training of such feature transformations. This has been done for direct feature transformations, for high dimensional polynomial features and for multilayer log-linear training. Multilayer log-linear transformation training makes the use of polynomial features of higher order computationally feasible. Another important aspect of this thesis is the discriminative splitting of log-linear models into log-linear mixture models with hidden variables. Thus the acoustic mixture models are trained as well as split during discriminative training, instead of using splitting information from a previous maximum likelihood training step. Recently, the speech recognition community has shifted its focus to deep neural networks for acoustic modelling. Thus we also apply the discriminative splitting concept to deep neural networks and achieve encouraging results