Parallel Gibbs Sampling for Hierarchical Dirichlet Processes via Gamma Processes Equivalence

The hierarchical Dirichlet process (HDP) is an intuitive and elegant technique to model data with latent groups. How-ever, it has not been widely used for practical applications due to the high computational costs associated with infer-ence. In this paper, we propose an effective parallel Gibbs sampling algorithm for HDP by exploring its connections with the gamma-gamma-Poisson process. Specifically, we develop a novel framework that combines bootstrap and Re-versible Jump MCMC algorithm to enable parallel variable updates. We also provide theoretical convergence analysis based on Gibbs sampling with asynchronous variable up-dates. Experiment results on both synthetic datasets and two large-scale text collections show that our algorithm can achieve considerable speedup as well as better inference ac-curacy for HDP compared with existing parallel sampling algorithms