The general memory neural network and its relationship with basis function architectures

A generalization of a class of neural network architectures based on a multiple quantization of input space combined with memory lookup operations is presented under the name of a general memory neural network (GMNN). Within this common framework it is shown that networks of this type are - for a variety of learning schemes - response-equivalent to basis function networks (i.e., radial basis function and kernel regression networks). In particular, this equivalence holds even if a GMNN does not employ explicit basis functions, which makes the architecture attractive from an implementational point of view and allows fast operation, both in the learning and response modes. Variants of the GMNN are discussed and examples of existing architectures conforming to this common framework are given. A generalization of a class of neural network architectures based on a multiple quantization of input space combined with memory lookup operations is presented under the name of a general memory neural network (GMNN). Within this common framework it is shown that networks of this type are - for a variety of learning schemes - response-equivalent to basis function networks (i.e., radial basis function and kernel regression networks). In particular, this equivalence holds even if a GMNN does not employ explicit basis functions, which makes the architecture attractive from an implementational point of view and allows fast operation, both in the learning and response modes. Variants of the GMNN are discussed and examples of existing architectures conforming to this common framework are given.