Learning of modular structured networks

AbstractLearning of large-scale neural networks suffers from computational cost and the local minima problem. One solution to these difficulties is the use of modular structured networks. Proposed here is the learning of modular networks using structural learning with forgetting. It enables the formation of modules. It also enables automatic utilization of appropriate modules from among the previously learned ones. This not only achieves efficient learning, but also makes the resulting network understandable due to its modular character.In the learning of a Boolean function, the present module acquires information from its subtask module without any supervision. In the parity problem, a previously learned lower-order parity problem is automatically used. The geometrical transformation of figures can be realized by a sequence of elementary transformations. This sequence can also be discovered by the learning of multi-layer modular networks. These examples well demonstrate the effectiveness of modular structured networks constructed by structural learning with forgetting