Embedding in Neural Networks: A-Priori Design of Hybrid Computers for Prediction

International audienceThe prediction of complex signals is among the most important applications of random recurrent Neural Networks (rRNN). Yet, no theory which completely describes prediction in rRNNs exists. As such, these systems remain "black boxes". Based on nearest neighbors theory and random nonlinear mapping, we fully describe the mechanisms employed by rRNNs solving this essential task. Our approach combines machine learning techniques (Reservoir Computing) and dynamical systems theory. We derive optimization cost functions which are (a) task specific, and (b) go far beyond the simple optimization of the prediction error. Going beyond, we demonstrate the consequences resulting from our theory. Based on our analysis of an rRNN stabilizing of an arrhythmic heart, we amend this system by a nonvolatile external FIFO memory. Furthermore, we are able a-priori specify the FIFO's properties, strongly reducing the previous training efforts of such systems