Printed in Great Britain Covariance structure of the Gibbs sampler with applications to the comparisons of estimators and augmentation schemes

We study the covariance structure of a Markov chain generated by the Gibbs sampler, with emphasis on data augmentation. When applied to a Bayesian missing data problem, the Gibbs sampler produces two natural approximations for the posterior distribution of the parameter vector: the empirical distribution based on the sampled values of the parameter vector, and a mixture of complete data posteriors. We prove that Rao-Blackwellization causes a one-lag delay for the autocovariances among dependent samples obtained from data augmentation, and consequently, the mixture approximation produces estimates with smaller variances than the empirical approximation. The covari-ance structure results are used to compare different augmentation schemes. It is shown that collapsing and grouping random components in a Gibbs sampler with two or three components usually result in more efficient sampling schemes