Approximate inference via variational sampling

We propose a new method to approximately integrate a function with respect to a given probability distribution when an exact computation is intractable. The method is called “variational sampling ” as it involves fitting a simplified distribution for which the integral has a closed-form expression, and using a set of randomly sampled control points to optimize the fit. The novelty lies in the chosen objective function, namely a Monte Carlo approximation to the generalized Kullback-Leibler divergence, which differs from classical methods that implement a similar idea, such as Bayesian Monte Carlo and importance sampling. We review several attractive mathematical properties of variational sampling, including well-posedness under a simple condition on the sample size, and a central limit theorem analogous to the case of importance sampling. We then report various simulations that essentially show that variational sampling has the potential to outperform existing methods within comparable computation time in estimating moments of order up to 2. We conclude with a brief discussion of desirable enhancements.