Divide-and-Conquer Learning and Modular Perceptron Networks

Abstract—A novel modular perceptron network (MPN) and divide-and-conquer learning (DCL) schemes for the design of modular neural networks are proposed. When a training process in a multilayer perceptron falls into a local minimum or stalls in a flat region, the proposed DCL scheme is applied to divide the current training data region (e.g., a hard to be learned training set) into two easier (hopely) to be learned regions. The learning process continues when a self-growing perceptron network and its initial weight estimation are constructed for one of the newly partitioned regions. Another partitioned region will resume the training process on the original perceptron network. Data region partitioning, weight estimating and learning are iteratively repeated until all the training data are completely learned by the MPN. We have evaluated and compared the proposed MPN with several representative neural networks on the two-spirals problem and real-world dataset. The MPN achieves better weight learning performance by requiring much less data presentations (99.01% 87.86 % lesser) during the network training phases, and better generalization performance (4.0 % better), and less processing time (2.0 % 81.3 % lesser) during the retrieving phase. On learning the real-world data, the MPN’s show less overfitting compared to single MLP. In addition, due to its self-growing and fast local learning characteristics, the modular network (MPN) can easily adapt to on-line and/or incremental learning requirements for a rapid changing environment. Index Terms—Divide-and-conquer learning, modular percep-tron network, multilayer perceptron, weight estimation. I