Hilbert space embeddings of hidden markov models

Hidden Markov Models (HMMs) are impor-tant tools for modeling sequence data. How-ever, they are restricted to discrete latent states, and are largely restricted to Gaussian and discrete observations. And, learning al-gorithms for HMMs have predominantly re-lied on local search heuristics, with the ex-ception of spectral methods such as those de-scribed below. We propose a nonparamet-ric HMM that extends traditional HMMs to structured and non-Gaussian continuous dis-tributions. Furthermore, we derive a local-minimum-free kernel spectral algorithm for learning these HMMs. We apply our method to robot vision data, slot car inertial sensor data and audio event classification data, and show that in these applications, embedded HMMs exceed the previous state-of-the-art performance. 1