On the problems of neural networks with multi-state neurons

For realistic neural network applications the storage and recognition of gray-tone patterns, i.e., patterns where each neuron in the network can take one of $Q$ different values, is more important than the storage of black and white patterns, although the latter has been more widely studied. Recently, several groups have shown the former task to be problematic with current techniques since the useful storage capacity, $\alpha$, generally decreases like: $\alpha\sim Q^{-2}$. In this paper one solution to this problem is proposed, which leads to the storage capacity decreasing like: $\alpha\sim(\log_2 Q)^{-1}$. For realistic situations, where $Q=256$ this implies an increase of nearly four orders of magnitude in the storage capacity. The price paid, is that the time needed to recall a pattern increases like: $\log_2 Q$. This price can be partially offset by an efficient parallel program which runs at 1.4 Gflops on a 32 processor iPSC/860 Hypercube