Neural systems as nonlinear filters

Experimental data show that biological synapses behave quite differently from the symbolic synapses in all common articial neural network models. Biological synapses are dynamic, i.e., their weight changes on a short time scale by several hundred percent in dependence of the past input to the synapse. In this article we address the question how this inherent synaptic dynamics which should not be confused with long term learning affects the computational power of a neural network. In particular we analyze computations on temporal and spatio-temporal patterns, and we give a complete mathematical characterization of all lters that can be approximated by feedforward neural networks with dynamic synapses. It turns out that even with just a single hidden layer such networks can approximate a very rich class of nonlinear lters: all lters that can be characterized by Volterra series. This result is robust with regard to various changes in the model for synap-tic dynamics. Our characterization result provides for all nonlinear lters that are approximable by Volterra series a new complexity hierarchy which is related to the cost of implementing such lters in neural systems.