Speeding Up MCMC by Efficient Data Subsampling

<p>We propose subsampling Markov chain Monte Carlo (MCMC), an MCMC framework where the likelihood function for <i>n</i> observations is estimated from a random subset of <i>m</i> observations. We introduce a highly efficient unbiased estimator of the log-likelihood based on control variates, such that the computing cost is much smaller than that of the full log-likelihood in standard MCMC. The likelihood estimate is bias-corrected and used in two dependent pseudo-marginal algorithms to sample from a perturbed posterior, for which we derive the asymptotic error with respect to <i>n</i> and <i>m</i>, respectively. We propose a practical estimator of the error and show that the error is negligible even for a very small <i>m</i> in our applications. We demonstrate that subsampling MCMC is substantially more efficient than standard MCMC in terms of sampling efficiency for a given computational budget, and that it outperforms other subsampling methods for MCMC proposed in the literature. Supplementary materials for this article are available online.