Bayesian Techniques for Neural Networks - Review and Case Studies

We give a short review on Bayesian techniques for neural networks and demonstrate the advantages of the approach in a number of industrial applications. Bayesian approach provides a principled way to handle the problem of overfitting, by averaging over all model complexities weighted by their posterior probability given the data sample. The approach also facilitates estimation of the confidence intervals of the results, and comparison to other model selection techniques (such as the committee of early stopped networks) often reveals faulty assumptions in the models. In this contribution we review the Bayesian techniques for neural networks and present comparison results from several case studies that include regression, classification, and inverse problems. 1 INTRODUCTION In non-linear function approximation and classification, neural networks have become popular tools in recent years. With neural networks the main difficulty is in controlling the complexity of the model. It is well ..