Some Contributions to Approximate Inference In Bayesian Statistics

Due to the availability of larger data-sets and the complexity of Bayesian statistical models in modern applications, the need for fast and approximate inference techniques is becoming more and more prevalent. In this thesis, we contribute three approaches to approximate Bayesian inference in statistics. Firstly, we present a sequential algorithm for fast fitting of Dirichlet process mixture (DPM) models. It provides a means for fast approximate Bayesian inference for mixture data which is useful when the data-sets are so large that many standard computational methods cannot be applied efficiently. This algorithm can be used in practice, to initially interrogate the data such that exact data analysis can be applied later on. The numerical results suggest that our algorithm can prove to be an efficient tool for fast, but approximate inference for DPMs. Secondly, we develop a new particle filter for approximating Feynman-Kac models with indicator potentials. Examples of such models include approximate Bayesian computation (ABC) posteriors associated with hidden Markov models (HMMs). We also use this approach to construct a new particle Markov chain Monte Carlo (PMCMC) algorithm for static parameter estimations associated to HMMs. Lastly, we construct an ABC approximation scheme of doubly intractable HMMs whose observation and transition densities are computational intractable. This scheme is facilitated by a new collapsed state space representation. We propose state-of-the-art Sequential Monte Carlo (SMC) algorithms for filtering via this ABC approximation. In addition, we can use these SMC algorithms within MCMC algorithms for batch static parameter inference for doubly intractable HMMs.Open Acces