Comparative aspects of neural network algorithms for on-line modelling of dynamic processes

This paper reviews the model structures and learning rules of four commonly used artificial neural networks: the Cerebellar Model Articulation Controller, B-Splines, Radial Basis Functions and Multilayered Perceptron networks. Their dynamic modeling abilities are compared using a two-dimensional nonlinear noisy time series. The network performances are evaluated based on their network surface plots, phase/time history plots, learning curves, prediction error autocorrelation functions, and finally their short-range prediction error variances. The modeling results suggest that all four networks were able to capture the underlying dynamics of the time series. Also, specific prior knowledge about the time series was incorporated into the B-Splines model, and is used to highlight an important trade-off between the model flexibility and high-dimensional modeling ability in the B-Splines and CMAC networks. In general, when the network model is well-conditioned and linear with respect to its adaptable parameters, simpler on-line learning rules often provide adequate convergence properties. Alternatively, when the model is highly nonlinear, complicated learning rules which utilize high-order gradient information are generally required at the expense of increased computational complexity