On the adversarial robustness of Gaussian processes

We study the robustness of Bayesian inference with Gaussian processes (GP) under adversarial attack settings. We begin by noticing how the distinction between model prediction and decision in Bayesian settings naturally leads us to give two different notions of adversarial robustness. The former, probabilistic adversarial robustness, is concerned with the behaviour of the posterior distribution, formally characterising its worst-case attack uncertainty. On the other hand, adversarial robustness is concerned with local stability of the model decision, and is strictly correlated with bounds on the predictive posterior distribution of the model. In the first part of this thesis we show how, by relying on the Borell-TIS inequality, the computation of probabilistic adversarial robustness can be translated to the solution of a set of optimisation problems defined over the GP posterior mean, variance and specific derived quantities. In order to solve these, we develop a general framework for the lower- and upper-bounding of GP posterior parameters, which relies on interval bound propagation techniques, the computation of linear and lower upper bounding functions and the solution of linear and convex-quadratic programming problems. Employing the central limit theorem for stochastic processes, we then demonstrate how the derived bounds can also be used for the adversarial analysis of infinitely-wide deep BNN architectures. In the second part of this thesis, we show how a suitably defined discretisation of the GP latent space can be used to convert the computation of adversarial robustness to the solution of a finite number of optimisation problem over a set of uni-dimensional Gaussian integral functions. We proceed by extending and adapting the GP optimisation framework developed in the context of probabilistic robustness to the formal solution of these integrals. We rely on the theory of branch-and-bound optimisation algorithms to formally prove that our method is guaranteed to terminate in finitely-many steps to an $\epsilon$-exact solution of the problem, for any $\epsilon > 0$ selected a-priori. Furthermore, the method developed is anytime, in that it can be stopped at any point during its computation and still provide formal lower and upper bounds that can be used to certify the GP adversarial robustness. By carefully designing suitable prior functions, we then show how GPs provide us with competitive and state-of-the-art models for their application in affective computing. We finally rely on three datasets for affect recognition from physiological signals as a real-world testbed to analyse the scalability and the practical feasibility of the methods we have developed for the verification and interpretation of GP models, which we argue are crucial for the development of machine learning systems that have to interact with humans in clinical situations.