Markov chain Monte Carlo and variational inference: Bridging the gap

Recent advances in stochastic gradient variational inference have made it possi-ble to perform variational Bayesian inference with posterior approximations con-taining auxiliary random variables. This enables us to explore a new synthesis of variational inference and Monte Carlo methods where we incorporate one or more steps of MCMC into our variational approximation. We describe the theoretical foundations that make this possible and show some promising first results. 1 Stochastic gradient variational inference At the center of Bayesian analysis is the posterior distribution p(z|x), where z is a set of unknown parameters or latent variables and x is the observed data. If the prior p(z) and likelihood p(x|z) have been specified, the posterior distribution can be computed using Bayes ’ rule. In practice this computation is often intractable and we have to resort to approximation methods. One such approx-imation method is variational inference, which casts inference as an optimization problem where we introduce a parameterized posterior approximation qθ(z|x) (or qθ(z)) which is fit to the posterior distribution by choosing its parameters θ to maximize the lower bound of the marginal likelihood