Mean Field Theory of Learning in Pruned Perceptrons

. We consider the mean field theory of optimally pruned perceptrons. Using the cavity method, microscopic equations for the weights and the examples are derived. Their statistical properties agree with previous results using the replica method. There is a gap in the weight distribution, causing an instability in the ground state. A rough energy landscape better describes the learning problem. Solutions to the microscopic equations result in high stability of the examples. 1 Introduction The mean field theory is useful in offering a microscopic description of learning processes in neural networks [1, 2, 3]. Large neural networks are mean field systems since the weights and examples strongly interact with each other during the learning process. This means that when one weight or example is considered, the influence of the rest of the system can be regarded as a background satisfying some averaged properties. Correlations between individual pairs or clusters of weights or examples can be..