Hierarchy and interpretability in neural models of language processing

Artificial neural networks have become remarkably successful on many natural language processing tasks. In this dissertation, I explore if these successes make them useful as explanatory models of human language processing. I focus in particular on hierarchical compositionality and recurrent neural networks (RNNs), which share with the human processing system the property that they process language temporally and incrementally. I consider two questions: i) Are RNNs in fact capable of processing hierarchical compositional structures (behavioural similarity)? ii) How can we obtain insight in how they do so (model interpretability)? I address these questions in six chapters, divided into three parts. In part 1, I consider artificial languages, which provide a clean setup in which processing of structure can be studied in isolation. In this part, I also introduce diagnostic classification -- an interpretability technique that plays an important role in this dissertation -- and reflect upon what it means for a model to be able to process hierarchical compositionality. In part two, I consider language models trained on naturalistic data (English sentences). Such models have been shown to capture syntax-sensitive long-distance subject-verb relationships. I investigate how they do so. I present detailed analyses of their inner dynamics, using diagnostic classification, neuron ablation and generalised contextual decomposition. Lastly, in the final part of this dissertation, I consider if a model's solution can be changed through an adapted learning signal. I use several of the previously introduced techniques to analyse the impact of adding this learning signal. In summary, in this dissertation I present many different analyses concerning the abilities of RNNs to process hierarchical structure, as well as several techniques to open these blackbox models. Overall, the results sketch a positive picture of the usefulness of such models as explanatory models of processing languages with hierarchical compositional semantics