Monte Carlo hidden Markov models

We present a learning algorithm for hidden Markov models with continuous state and observa-tion spaces. All necessary probability density functions are approximated using samples, along with density trees generated from such samples. A Monte Carlo version of Baum-Welch (EM) is employed to learn models from data, just as in regular HMM learning. Regularization during learning is obtained using an exponential shrinking technique. The shrinkage factor, which deter-mines the effective capacity of the learning algorithm, is annealed down over multiple iterations of Baum-Welch, and early stopping is applied to select the right model. We prove that under mild assumptions, Monte Carlo Hidden Markov Models converge to a local maximum in likeli-hood space, just like conventional HMMs. In addition, we provide empirical results obtained in a gesture recognition domain, which illustrate the appropriateness of the approach in practice