Gaussian processes for Bayesian classification via hybrid Monte Carlo

d.barber~aston.ac.uk c.k.i.williams~aston.ac.uk The full Bayesian method for applying neural networks to a pre-diction problem is to set up the prior/hyperprior structure for the net and then perform the necessary integrals. However, these inte-grals are not tractable analytically, and Markov Chain Monte Carlo (MCMC) methods are slow, especially if the parameter space is high-dimensional. Using Gaussian processes we can approximate the weight space integral analytically, so that only a small number of hyperparameters need be integrated over by MCMC methods. We have applied this idea to classification problems, obtaining ex-cellent results on the real-world problems investigated so far.