Interpretable Models in Probabilistic Machine Learning

This thesis describes contributions to the field of interpretable models in probabilistic machine learning, by first outlining the desiderata and properties associated with the term interpretability. We claim that probabilistic models are suitable candidates for interpretable machine learning, and this claim is supported by examples of such models that satisfy two key properties of interpretability: transparency, that offers an understanding of the model's mechanism, and post-hoc interpretability, that gives other useful information about the model after training, such as explaining its predictions. Henceforth, we introduce relevant background literature in probabilistic machine learning, focusing on Bayesian inference of probabilistic models. Armed with these pre-requisites, we proceed to describe examples of probabilistic models that enjoy various interpretable properties. First, we propose a method for regression that is motivated from Gaussian Processes (GPs), that has applications to collaborative filtering with side-information and generalises classic probabilistic matrix factorisation methods in this context. Second, we develop a scalable algorithm for automated GP model selection, whereby the form of the selected GP models allows them to be translated into a natural language description of its properties. Third, we introduce a Variational Autoencoder (VAE) model that can disentangle independent factors of variations in a dataset of images by learning a factorisable latent distribution in an unsupervised fashion. Finally, we describe a model that can learn stochastic processes in a data-driven fashion with deep architectures by using the concept of attention