Dynamic recurrent neural networks

We survey learning algorithms for recurrent neural networks with hidden units and attempt to put the various techniques into a common framework. We discuss xpoint learning al-gorithms, namely recurrent backpropagation and deterministic Boltzmann Machines, and non- xpoint algorithms, namely backpropagation through time, Elman's history cuto nets, and Jordan's output feedback architecture. Forward propagation, an online technique that uses adjoint equations, is also discussed. In many cases, the unied presentation leads to generalizations of various sorts. Some simulations are presented, and at the end, issues of computational complexity are addressed