Dynamics of neural networks and disordered spin systems

I obtain a number of results for the dynamics of several disordered spin systems, of successively greater complexity. I commence with the generalised Hopfield model trained with an intensive number of patterns, where in the thermodynamic limit macroscopic, deterministic equations of motion can be derived exactly for both the synchronous discrete time and asynchronous continuous time dynamics. I show that for symmetric embedding matrices Lyapunov functions exist at the macroscopic level of description in terms of pattern overlaps. I then show that for asymmetric embedding matrices several types of bifurcation phenomena to complex non-transient dynamics occur, even in this simplest model. Extending a recent result of Coolen and Sherrington, I show how the dynamics of the generalised Hopfield model trained with extensively many patterns and non-trivial embedding matrix can be described by the evolution of a small number of overlaps and the disordered contribution to the 'energy', upon calculation of a noise distribution by the replica method. The evaluation of the noise distribution requires two key assumptions: that the flow equations are self averaging, and that equipartitioning of probability occurs within the macroscopic sub-shells of the ensemble. This method is inexact on intermediate time scales, due to the microscopic information integrated out in order to derive a closed set of equations. I then show how this theory can be improved in a systematic manner by introducing an order parameter function - the joint distribution of spins and local alignment fields, which evolves in time deterministically, according to a driven diffusion type equation. I show how the coefficients in this equation can be evaluated for the generalised Sherrington-Kirkpatrick model, both within the replica symmetric ansatz, and using Parisi's ultrametric ansatz for the replica matrices, upon making once again the two key assumptions (self averaging and equipartitioning). Since the order parameter is now a continuous function, however, the assumption of equipartitioning within the macroscopic sub-shells is much less restricting.