Argument Diagramming and Diagnostic Reliability

Diagrammatic models of argument are increasingly prominent in AI and Law. Unlike everyday language these models formalize many of the the compo-nents and relationships present in arguments and permit a more formal analysis of an arguments ’ structural weaknesses. Formalization, however, can raise problems of agreement. In order for argument diagramming to be widely accepted as a com-munications tool, individual authors and readers must be able to agree on the qual-ity and meaning of a diagram as well as the role that key components play. This is especially problematic when arguers seek to map their diagrams to or from more conventional prose. In this paper we present results from a grader agreement study that we have conducted using LARGO diagrams. We then describe a detailed ex-ample of disagreement and highlight its implications for both our diagram model and modeling argument diagrams in general