Grammar Transfer in a Second Order Recurrent Neural Network

It has been known that people, after being exposed to sentences generated by an artificial grammar, acquire implicit grammatical knowledge and are able to transfer the knowledge to inputs that are generated by a modified grammar. We show that a second order recurrent neural network is able to transfer grammatical knowledge from one language (generated by a Finite State Machine) to another language which differ both in vocabularies and syntax. Representa-tion of the grammatical knowledge in the network is analyzed using linear discriminant analysis.