Exploiting prior knowledge in machine learning model design

The design of models which are appropriate for specific tasks is an important activity in machine learning. This thesis considers multiple ways in which knowledge about the task at hand can be incorporated into the design of a machine learning model: (i) by using Bayesian models, which incorporate prior knowledge in probability distributions; (ii) by designing models to respect the symmetries of the task; (iii) by adapting models in a practical setting. For Bayesian models, we propose a probabilistic numeric method for computing integrals which arise in the context of Bayesian inference. Respecting the symmetries of a task comes under the framework of group equivariance and invariance. We provide a theoretical investigation of Deep Sets, a prominent permutation-invariant model, and propose an extension of the SE(3)-Transformer, an SE(3)-equivariant model. Finally, we present an investigation of the application of machine learning techniques to problems in atmospheric physics, in which the relevant models are adapted to specific known features of the problem.