Stochastic Variational Inference for Bayesian Time Series Models

Bayesian models provide powerful tools for an-alyzing complex time series data, but perform-ing inference with large datasets is a challenge. Stochastic variational inference (SVI) provides a new framework for approximating model poste-riors with only a small number of passes through the data, enabling such models to be fit at scale. However, its application to time series models has not been studied. In this paper we develop SVI algorithms for several common Bayesian time series models, namely the hidden Markov model (HMM), hid-den semi-Markov model (HSMM), and the non-parametric HDP-HMM and HDP-HSMM. In ad-dition, because HSMM inference can be expen-sive even in the minibatch setting of SVI, we de-velop fast approximate updates for HSMMs with durations distributions that are negative binomi-als or mixtures of negative binomials. 1