Multi-Layer Perceptrons: Robustness and the evolution of the weights through time

We consider the behaviour of the MLP in the presence of gross outliers in the training data. We show, via a consideration of the "influence curve", that the non-- robust behaviour of the MLP is due to the uncontrolled growth in the absolute magnitude of the weights, and suggest imposing a length constraint on the weights vector, for example by transforming the weights to polar coordinates, as a remedy to this. 1 The MLP classifier The multi--layer perceptron (MLP) is a powerful (in terms of the class of functions that can be approximated) distribution--free regression method. However, this very power means that the method is susceptible to over--fitting and to "modeling noise" in the training data. In order to rectify this tendency, a number of methods have been tried, including: 1. adding noise to the training data (jittering); 2. scaling the weights down after training; 3. replacing the target value with the convolution of the target and a probability distribution; 4. adding a regul..