Learning of Associative Memory Networks by Penalty Methods

This paper concerns the learning of associative memory networks. We derive inequality associative conditions for stored patterns in the network. Under these associative conditions, we find regions each of which is mapped by the network function into a neighbor of an associative pattern. To make large the regions, a functional is derived using their shape. The functional is minimized under the inequality associative conditions. We show that this minimization problem has a unique solution, and solve the problem by combining the penalty methods with the gradient methods. This solving process gives a learning algorithm for associative networks. Our theory is first used to analyze two-layer autoassociative networks. It is shown that the network function becomes a contraction mapping in each of the regions derived under inequality autoassociative conditions. We also show that the function has a fixed point extremely near a stored pattern. This implies that the region obtained is a domain of attraction and that the fixed point is its attractor. Next, our learning algorithm is applied to make a heteroassociative network which is useful for solving classification problems. By adding one more layer to the network, we construct a three-layer autoassociative network whose input-output function is shown to be a contraction mapping in some domains. In simulations, efficiency of our two autoassociative networks is verified in character recognition