Learning long sequences in binary neural networks

Abstract—An original architecture of oriented sparse neural networks that enables the introduction of sequentiality in as-sociative memories is proposed in this paper. This architecture can be regarded as a generalization of a non oriented binary network based on cliques recently proposed. Using a limited neuron resource, the network is able to learn very long sequences and to retrieve them from only the knowledge of any sequence of consecutive symbols. Index Terms—oriented neural network; learning machine; associative memory; sparse coding; directed graph; sequential learning; efficiency. I