Modularity within Neural Networks

Module Figure 7: Network of Autoassociative Modules There are several advantages exhibited by a modular network of such components. Firstly, the processing of the network is clearer to understand since the internal structure of the entire network is more open to examination than the single hidden layer in a standard back-propagation network. Secondly, the processing is more flexible since the component modules can be used to provide input to several modules, such as in a task which requires several processing streams. Thirdly, the network may also have a benefit in complexity and performance because of the compactness of the representations used. If multiple input sources are considered together in a single network, the crucial differences between data patterns may be difficult to learn because of their relatively small impact on the entire vector (e.g., a difference of only a few bits between two 1000-bit long vectors will tend to be difficult to notice). On the other hand, in a modu..