Creating an argumentation corpus: do theories apply to real arguments? A case study on the legal argumentation of the ECHR

Argumentation annotation is a crucial step in applying machine learning techniques to the argumentation field. However, there exist few argumentation corpora and their development has not been studied in depth. In this paper we present a study conducted during the creation of a legal argumentation corpus. It shows how well-known argumentation theories are used as the background framework of the annotation process and which difficulties are found when applying those theories to real argumentation. The aim of the paper is to highlight different critical points humans encounter when applying theory to real argumentation, allowing better and faster approaches in future annotation processes. Furthermore, we also highlight fundamental problems of the chosen argumentation theories and thereby offer ideas for future research on argumentation theory.status: publishe