Variational Bayesian Analysis for Hidden Markov Models

The variational approach to Bayesian inference enables simultaneous estimation of model parameters and model complexity. An interesting feature of this approach is that it also leads to an automatic choice of model complexity. Empirical results from the analysis of hidden Markov models with Gaussian observation densities illustrate this. If the variational algorithm is initialised with a large number of hidden states, redundant states are eliminated as the method converges to a solution, thereby leading to a selection of the number of hidden states. In addition, through the use of a variational approximation, the Deviance Information Criterion for Bayesian model selection can be extended to the hidden Markov model framework. Calculation of the Deviance Information Criterion provides a further tool for model selection which can be used in conjunction with the variational approach