A New Scheme for Incremental Learning

We present a new incremental procedure for supervised learning with noisy data. Each step consists in adding to the current network a new unit which is trained to learn the error of the network. The incremental step is repeated until the error of the current network can be considered as a noise. The stopping criterion is very simple and can be directly deduced from a statistical test on the estimated parameters of the new unit. First experimental results point out the efficacy of this new incremental scheme. Current works deal with theoretical analysis and practical refinements of the algorithm. 1 Introduction One of the problems encountered when dealing with neural networks is to specify their architecture: number of layers, number of units per hidden layer. Considering a network with a too large number of units certainly leads to an oversized and bad-generalizing network. An appropriate solution would be to use an incremental procedure which adjusts the size of the network to the pr..