Learning behavior and temporary minima of two-layer neural networks

Abstract--This paper presents a mathematical nalysis of the occurrence of temporary minima during training of a single-output, wo-layer neural network, with learning according to the back-propagation algorithm. A new vector decomposition method is introduced, which simplifies the mathematical nalysis of learning of neural networks considerably. The analysis hows that temporary minima are inherent o multilayer networks learning. A number of numerical results illustrate the analytical conclusions