Self-Modifying Dynamical Systems for Reservoir Computing

Biology has always been an inspiration in the quest for artificial intelligence. By combining a hierarchy of chemistry, cells, and structures, a biological system able to learn, adapt and perform complex problem solving can emerge. Reservoir Computing (RC) is a highly efficient bio-inspired technique for working with time dependent data. Reservoir computing utilises an untrained recurrent dynamical system as a reservoir of dynamics. A single readout layer can then be trained to correlate the state of the reservoir to some target value. One aspect of biological systems that are of great interest is their ability to self-organise. Self-Modifying Cartesian Genetic Programming (SMCGP) is a genetic programming algorithm that mimics these traits through self-organising topology. This thesis investigates the use of SMCGP as a reservoir of dynamics in a reservoir computing system (SMCGP-RC). Reservoir Computing is a general framework to design, train and analyse recurrent neural networks. However, any high dimensional dynamic system that can be stimulated by input, has the right dynamic properties, and has readable state, can be used as a reservoir. This sets few constraints on possible implementations. Both digital and physical reservoirs have been widely researched, ranging from Random Boolean Networks (RBN) to a bucket filled with water. SMCGP is a graph-based genetic programming algorithm based on Cartesian Genetic Programming (CGP). Including the mathematical operators in CGP, programs produced by SMCGP contain self-modification operators. These operators enable bio-inspired qualities such as self-regulating structures, scalability, adaptation, and self-repair. The SMCGP-RC implementation presented in this thesis utilises an evolutionary algorithm to search for SMCGP programs where the topology is continuously changing. These programs are then evaluated against common benchmarking problems. The results show that the dynamics of SMCGP programs with continuous topological changes can be harnessed for computation in an RC environment. The temporal parity experiments presented show that when output from the previous time step is made available for SMCGP programs, they can exhibit the necessary dynamic properties to solve non-linear temporal problems