Memory Capacity of a Neural Network

We propose to measure the memory capacity of a state machine by the numbers of discernible states, where two states are defined to be discernible if the machine manifests the identical input-output mapping in both states. According to the definition, a neuron network of n\u3e0 inputs and one output, with an uncountable set of internal states, has the memory capacity of log2TF(n), where TF(n) is the number of different Boolean functions the network can realize with different synaptic weight and threshold values. It is shown that such a network with k\u3e0 linear threshold units can realize at most 2k(n2+k2) Boolean functions and therefore the network has memory capacity of at most k(n2+k2) bits or 2k3 bits when