Collective Learning Generally Overcomes Local Optima and Converges to the Global Optimum

Local minima represent a major problem for neural network learning procedures. In this article we present a new procedure, collective learning, that leads to improved global convergence. We have tested our procedure on several neural networks and on the multimodal functions proposed by De Jong and Rastrigin. In our tests we have reached a success ratio of 100 %. In addition we give a few remarks on the theorie of collective learning and give an estimate of the convergence behavior as well. Moreover, our procedure is very fast. 1 INTRODUCTION As is well-known, getting stuck in local minima is a major problem of learning in neural networks and other kinds of optimization procedures. For the problems of interest one defines a fitness function (also called error or objective function) which is to be optimized. With respect to the fitness function, the term local minimum means that the fitness value is far from a known global value but no progress in the optimization process is attainable. ..