The echo index and multistability in input-driven recurrent neural networks

This is the author accepted manuscript. The final version is available from Elsevier via the DOI in this record.A recurrent neural network (RNN) possesses the echo state property (ESP) if, for a given input sequence, it “forgets” any internal states of the driven (nonautonomous) system and asymptotically follows a unique, possibly complex trajectory. The lack of ESP is conventionally understood as a lack of reliable behaviour in RNNs. Here, we show that RNNs can reliably perform computations under a more general principle that accounts only for their local behaviour in phase space. To this end, we formulate a generalisation of the ESP and introduce an echo index to characterise the number of simultaneously stable responses of a driven RNN. We show that it is possible for the echo index to change with inputs, highlighting a potential source of computational errors in RNNs due to characteristics of the inputs driving the dynamics.Engineering and Physical Sciences Research Council (EPSRC)Canada Research Chairs programNZ Marsden fun