Efficient Learning in Sparsely Connected Boltzmann Machines

We present a heuristical procedure for efficient estimation of the partition function in the Boltzmann distribution. The resulting speed-up is of immediate relevance for the speed-up of Boltzmann Machine learning rules, especially for networks with sparse connectivity. 1 Introduction Boltzmann Machines (BMs) [1] form an attractive group of Neural Networks for several reasons. The local learning rule, for instance, offers the possibility of parallel implementation. Their main disadvantage, however, is that computing the correlations hS i S j i exactly can only be done in a reasonable time for small networks. Although the correlations can be approximated with simulated annealing, this is very slow. Some good results have been reported about mean field learning, which speeds up the computation by approximating hS i S j i by hS i i hS j i. However, in most situations mean field learning leads to large errors and cannot be applied successfully. An approach which leads to tractable Boltzm..