Annotation of a Dutch Essay Corpus with Argument Structures and Quality Indicators

Based on the availability of previously annotated text corpora, the technique of argument mining (AM) aims to discover components in texts belonging to an argumentation structure. Due to the lack of such annotated corpus for the Dutch language, this paper presents a Dutch essay corpus with annotations of argumentation structures and quality indicators. We applied the annotation schemes and guidelines derived from previous studies to capture the argument structures of Dutch argumentative essays by identifying and classifying the argument components into major claims, claims, and premises as well as the support/attack relations between the components. Furthermore, we annotated persuasiveness scores and attributes that influence persuasiveness as quality indicators. The annotation task was performed by four native Dutch teachers who annotated 30 student-written Dutch argumentative essays. The inter-rater agreement of the annotations was generally lower compared to similar previous work, due to the less rigid format of the essays in our corpus and more annotators participating in the annotation task. However, the essays in our corpus are more in line with non-worked real-world text examples. To ensure the accuracy, objectivity, and reliability of the corpus a consolidation procedure was applied to the final compilation. This corpus presents a novel and reliable resource for future applications in argument mining tasks in the Dutch context. The corpus is publicly available via GitHub