Development of A New Recurrent Neural Network Toolbox (RNN-Tool) ∗ A Course Project Report on Training Recurrent Multilayer Perceptron and Echo State Network

In this report, we developed a new recurrent neural network toolbox, including the recurrent multilayer perceptron structure and its companying extended Kalman filter based training algorithms: BPTT-GEKF and BPTT-DEKF. Besides, we also constructed programs for designing echo state network with single reservoir, together with the offline linear regression based training algorithm. We name this toolbox as the RNN-Tool. Within the toolbox, we implement the RMLP and ESN as MATLAB structures, which are used throughout the processes of network generation, training and testing. Finally we study a predictive modeling case of a phase-modulated sinusoidal function to test this toolbox. Simulation results show that ESN can outperform the BPTT-GEKF and BTPP-DEKF methods both on computational load and prediction accuracy