An Overview of Temporal Backpropagation

This paper presents a derivation of a training algorithm for backpropagation neural networks which operate upon time-dependent input data, called "temporal backpropagation." The derivation uses transformation rules to generate the algorithm directly from the standard backpropagation algorithm. Issues encountered when training time-delay neural networks are discussed, and a number of examples are shown. 1 Introduction to Temporal Learning Neural networks have been used to solve problems in the field of pattern recognition and system identification with a fair amount of success. The backpropagation algorithm caused a renewal of interest in the field by providing a way of training multilayer networks with nonlinear transfer functions. The basic backpropagation algorithm assumes that the neural network is a combinational circuit, providing an output for a given input. However, many applications suitable for adaptive learning have inherently temporal structures, of which speech recognition..