Soft-decision decoding of a family of nonlinear codes using a neural network

We demonstrate the use of a continuous Hopfield neural network as a K-WinnerTake-All (KWTA) network. We prove that, given an input of N real numbers, such a network will converge to a vector of K positive one components and (N-K) negative one components, with the positive positions indicating the K largest input components. In addition, we show that the [(N K)] such vectors are the only stable states of the system. One application of the KWTA network is the analog decoding of error-correcting codes. We prove that the KWTA network performs optimal decoding. We consider decoders that are networks with nodes in overlapping, randomly placed KWTA constraints and discuss characteristics of the resulting codes. We present two families of decoders constructed by overlapping KWTA constraints in a structured fashion on the nodes of a neural network. We analyze the performance of these decoders in terms of error rate, and discuss code minimum distance and information rate. We observe that these decoders perform near-optimal, soft-decision decoding on a class of nonlinear codes. We present a gain schedule that results in improved decoder performance in terms of error rate. We present a general algorithm for determining the minimum distance of codes defined by the stable states of neural networks with nodes in overlapping KWTA constraints. We consider the feasibility of embedding these neural network decoders in VLSI technologies and show that decoders of reasonable size could be implemented on a single integrated circuit. We also analyze the scaling of such implementations with decoder size and complexity. Finally, we present an algorithm, based on the random coding theorem, to communicate an array of bits over a distributed communication network of simple processors connected by a common noisy bus