A Synthesis of Gradient and Hamiltonian Dynamics Applied to Learning in Neural Networks

The process of model learning can be considered in two stages: model selection and parameter estimation. In this paper a technique is presented for constructing dynamical systems with desired qualitative properties. The approach is based on the fact that an n-dimensional nonlinear dynamical system can be decomposed into one gradient and (n \Gamma 1) Hamiltonian systems. Thus, the model selection stage consists of choosing the gradient and Hamiltonian portions appropriately so that a certain behavior is obtainable. To estimate the parameters, a stably convergent learning rule is presented. This algorithm is proven to converge to the desired system trajectory for all initial conditions and system inputs. This technique can be used to design neural network models which are guaranteed to solve certain classes of nonlinear identification problems. Key Words Dynamical systems, System Identification Acknowledgments This research was supported by a grant from Boeing Computer Services under C..