Learning a Deterministic Finite Automaton With a Recurrent Neural Network

We consider the problem of learning a finite automaton with recurrent neural networks, given a training set of sentences in a language. We train Elman recurrent neural networks on the prediction task and study experimentally what these networks learn. We found that the network tends to encode an approximation of the minimum automaton that accepts only the sentences in the training set. 1 Introduction 1.1 The problem of inducing a deterministic finite automaton (DFA) The interest in DFA inference is partly induced from the larger goal of explaining how humans learn the grammar rules of their native language. There have been debates on whether children learn in an unsupervised mode, just by listening to other language speakers, or if they have innate knowledge of language. Therefore, it is an interesting problem to see what can be learned just by "listening to others", that is, from a set of grammatically correct sentences. While the complex syntactic rules of natural language cannot b..