On solving Ordinary Differential Equations using Gaussian Processes

We describe a set of Gaussian Process based approaches that can be used to solve non-linear Ordinary Differential Equations. We suggest an explicit probabilistic solver and two implicit methods, one analogous to Picard iteration and the other to gradient matching. All methods have greater accuracy than previously suggested Gaussian Process approaches. We also suggest a general approach that can yield error estimates from any standard ODE solver. 1 The Initial Value Problem Given an Ordinary Differential Equation (ODE) with known initial condition x(t1) = x1 d dt x(t) = f(t, x(t), θ) (1) the Initial Value Problem (IVP) is to find the differentiable function x(t) over some specified time interval t ∈ [t1, tT] that satisfies the ODE subject to the initial value condition. In general x(t) is a vector so that higher order scalar ODEs can be embedded as first order vector ODEs [8]. In general this problem requires an approximate numerical solution and we denote the approximation at time tn to x(tn) by xn. There is a vas