Sampling statistical distributions in physics: a machine learning approach

This thesis presents work which uses Machine Learning techniques in a variety of sampling situations which appear in physics. In the first Chapter some background on Machine Learning will be presented which will lay the foundations required for the later Chapters. Next we will look at how a specific Machine Learning model, the Restricted Boltzmann Machine, can be trained to approximate a target distribution from data which has already been sampled from the target distribution. We estimate observables on states sampled from trained models and compare them to observables estimated directly from the training data. We present a technique for estimating the likelihood function of the model using annealed importance sampling. Finally we present a closed form expression for extracting the N-point interactions which the model learns from the data directly from the parameters of the model, a result which is useful for a range of fields which study binary data. In the next Chapter we investigate a different generative model, the normalizing flow, and investigate its efficacy of generating configurations for a lattice scalar field theory. An initial study which quantifies how the cost of training this model scales with the system size is performed. Whilst the cost of training our models is significantly less than those reported in the proof of principle study which first presented using these models for this purpose [1], we discuss how there is still an exponential scaling of the training cost with the system size which must be overcome in order for these models to be practically useful. Finally we investigate inverse problems from a Bayesian perspective. With this framework, we are faced with the task of sampling from the posterior distributions in model space given the data. An approach for sampling in model space presented by the NNPDF collaboration is examined within this formal framework. We present some statistical estimators which can be used to validate a methodology which produces a sample of models. These estimators can be implemented in a closure test [2], where the data is artificially generated from a pre-existing underlying law. We show how these estimators can be used to check that the model distribution is self consistent when the data is fluctuated according to its prior. A proof of principle example is presented by performing a closure test using the latest NNPDF methodology and we show that the NNPDF MC approach is successful at producing a sample of model replicas which have faithful uncertainties. Whilst these estimators are practical to implement and are shown to be useful in a non-trivial setting, we discuss the possibility of defining some estimators directly in model space which could give more general information on model uncertainties for a wide range of inverse problems