Towards automatically classifying case law citation treatment using neural networks

In common law legal systems, judges decide issues between parties (legal decision or case law) by reference to previous decisions that consider similar factual situations. Accordingly, these decisions typically feature rich citation networks, i.e., a new decision frequently cites previous relevant decisions (citation). These citations may, in varying degrees, express that a cited decision is applicable, not-applicable, or no longer current law. Such treatment label is important to a lawyer’s process of determining whether a case is proper law. These labels serve as a matter of convenience in citation indices enabling lawyers to prioritise decisions to examine to understand the current state of the law. They also prove useful in other areas such as prioritisation for manual summarisation of cases, where not all cases can be summarised, and automatic summarisation, or, potentially, as a ranking feature in case law retrieval. While a lawyer can determine the treatment of a cited case by reading a decision, this is time consuming and can increase legal costs. Currently, not all newly decided cases feature these treatment labels. Further, older cases typically do not. Given the large amount of new legal decisions decided each year, manual annotation of such treatment is not feasible. In this paper, we explore the effectiveness of neural network architectures for identifying case law citation treatment and importance (whether a case is important to a lawyer’s reasoning process). We find that these tasks are very difficult and various methods for text classification perform poorly. We address more comprehensively the task of citation importance for this reason while limiting our examination of the task of citation treatment to the modelling of the problem and the highlight of the intrinsic difficulty of the task. We make a test dataset available at github.com/ielab/caselaw-citations to stimulate further research that tackles this challenging problem. We also contribute a range of word embeddings learned over a large amount of processed case law text