Some System Theoretic Considerations when Learning Dynamical Systems with Neural Networks

In this paper we present a class of nonlinear neural network models and an associated learning algorithm that always converges to a set of network parameters (e.g., the connection weights) such that the error between the network trajectories and the desired trajectories vanishes, for all initial conditions and system inputs. Our models are the well known class of additive neural networks. We show that additive networks are one instance of the class of models whose dynamics can be decomposed into gradient and Hamiltonian portions. Furthermore, we use the Hamiltonian potential function to prove that additive networks possess bounded-input bounded-state stability for some minor restrictions on the node output functions. Also we present a condition for persistent excitation which guarantees that the parameter estimates converge to the actual parameter values. Since parameter convergence means that the model accurately reproduces the system outputs for all inputs, it is analogous to the not..