Analysis of the Storage Capacity of RAM-based Neural Networks

This paper presents a probabilistic approach based on collisions to assess the storage capacity of RAM-based neural networks. The analysis at neuron level provides the basis for evaluation of storage capacity in the architectures. The approach is tested in the GNU and pyramid networks. In the GNU as an auto-associative memory, the theoretical results fit well with Braga's experimental data and are more broadly applicable than Braga's and Wong & Sherrington's theoretical results. For the pyramid, the theoretical results fit well with Penny & Stonham's experimental data. We discuss the approximations and limitations of the approach. An important aspect of this approach is that the storage capacity of any network can be assessed for the specific data which it is going to deal with --- for any probability distribution. This is a tool to be used for "learning" the connections of RAM-based networks. Keywords: RAM-based neural networks, storage capacity, elementary probability. 1 Introductio..