Modelling Weight- and Input-Noise in MLP Learning

This paper presents a study of weight- and input-noise in feedforward network training algorithms. In theory for the optimal least-squares case noise can be modelled by a single cost function term. However we believe that such ideal conditions are uncommon in practice. Both first and second derivative terms are shown to have the potential to de-sensitize the trained network's outputs to weight- or input-corruption. Simulation experiments illustrate these points by comparing the ideal case with a more realistic real-world example. The results show that although the second derivative term can influence the network solution in the practical case, the first derivative term is dominant