Block-structured Recurrent Neural Networks

This paper introduces a new class of dynamic multi layer perceptrons, called Block Feedback Neural Networks (B F N). B F N have been developed to provide a systematic way to build networks of high complexity, including networks with coupled loops, nested loops and so on. B F Ns are specified using a block notation. Any B F N can be seen as a block and connected to other B F Ns using a fixed number of elementary connections. The result of such a connection can also be considered as a block, and connected to other blocks, in a recursive fashion. We develop a cost-minimizing supervised training algorithm for this model. The algorithm is a gradient-descent type, and is tailored on the block structure of the model. Finally, we present some experiments of use of B F Ns. 1 Introduction The interest in dynamic neural networks has been steadily growing in the past few years, and a number of models have been proposed -- such as Williams and Zipser's Forward Propagation network (Williams and Z..