On optimal nonlinear associative recall

The problem of determining the nonlinear function (“blackbox”) which optimally associates (on given criteria) two sets of data is considered. The data are given as discrete, finite column vectors, forming two matricesX (“input”) andY (“output”) with the same numbers of columns and an arbitrary numbers of rows. An iteration method based on the concept of the generalized inverse of a matrix provides the polynomial mapping of degreek onX by whichY is retrieved in an optimal way in the least squares sense. The results can be applied to a wide class of problems since such polynomial mappings may approximate any continuous real function from the “input” space to the “output” space to any required degree of accuracy. Conditions under which the optimal estimate is linear are given. Linear transformations on the input key-vectors and analogies with the “whitening” approach are also discussed. Conditions of “stationarity” on the processes of whichX andY are assumed to represent a set of sample sequences can be easily introduced. The optimal linear estimate is given by a discrete counterpart of the Wiener-Hopf equation and, if the key-signals are noise-like, the holographic-like scheme of associative memory is obtained, as the optimal nonlinear estimator. The theory can be applied to the system identification problem. It is finally suggested that the results outlined here may be relevant to the construction of models of associative, distributed memory