Inferring Stochastic Regular Grammars with Recurrent Neural Networks

. Recent work has shown that the extraction of symbolic rules improves the generalization performance of recurrent neural networks trained with complete (positive and negative) samples of regular languages. This paper explores the possibility of inferring the rules of the language when the network is trained instead with stochastic, positiveonly data. For this purpose, a recurrent network with two layers is used. If instead of using the network itself, an automaton is extracted from the network after training and the transition probabilities of the extracted automaton are estimated from the sample, the relative entropy with respect to the true distribution is reduced. 1 Introduction A number of papers [1, 2, 3, 4] have explored the ability of second-order recurrent neural networks to infer simple regular grammars from complete (positive and negative) training samples. The generalization performance of a finite state automaton extracted from the trained network is better than..