Computational Argumentation Quality Assessment in Natural Language

Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 176–187, Valencia, Spain, April 3-7, 2017. c⃝2017 Association for Computational Linguistics Computational Argumentation Quality Assessment in Natural Language Henning Wachsmuth Bauhaus-Universität Weimar Weimar, Germany henning.wachsmuth@uni-weimar.de Nona Naderi University of Toronto Toronto, Canada nona@cs.toronto.edu Yufang Hou IBM Research Dublin, Ireland yhou@ie.ibm.com Yonatan Bilu IBM Research Haifa, Israel yonatanb@il.ibm.com Vinodkumar Prabhakaran Stanford University Stanford, CA, USA vinod@cs.stanford.edu Tim Alberdingk Thijm, Graeme Hirst University of Toronto Toronto, Canada {thijm,gh}@cs.toronto.edu Benno Stein Bauhaus-Universität Weimar Weimar, Germany benno.stein@uni-weimar.de Abstract Research on computational argumentation faces the problem of how to automatically assess the quality of an argument or argu- mentation. While different quality dimen- sions have been approached in natural lan- guage processing, a common understand- ing of argumentation quality is still missing. This paper presents the first holistic work on computational argumentation quality in natural language. We comprehensively sur- vey the diverse existing theories and ap- proaches to assess logical, rhetorical, and dialectical quality dimensions, and we de- rive a systematic taxonomy from these. In addition, we provide a corpus with 320 ar- guments, annotated for all 15 dimensions in the taxonomy. Our results establish a com- mon ground for research on computational argumentation quality assessment. 1 Introduction What is a good argument? What premises should it be based on? When is argumentation persuasive? When is it reasonable? We subsume such ques- tions under the term argumentation quality; they have driven logicians, rhetoricians, linguists, and argumentation theorists since the Ancient Greeks (Aristotle, 2007). Now that the area of computa- tional argumentation is seeing an influx of research activity, the automatic assessment of argumentation quality is coming into the focus, due to its impor- tance for envisioned applications such as writing support (Stab and Gurevych, 2014) and argument search (Wachsmuth et al., 2017), among others. Existing research covers the mining of argument units (Al-Khatib et al., 2016), specific types of evi- dence (Rinott et al., 2015), and argumentative rela- tions (Peldszus and Stede, 2015). Other works clas- sify argumentation schemes (Feng et al., 2014) and frames (Naderi and Hirst, 2015), analyze overall argumentation structures (Wachsmuth et al., 2015), or generate claims (Bilu and Slonim, 2016). Also, theories of argumentation quality exist, and some quality dimensions have been assessed computa- tionally (see Section 2 for details). Until now, how- ever, the assertion of O’Keefe and Jackson (1995) that there is neither a general idea of what consti- tutes argumentation quality in natural language nor a clear definition of its dimensions still holds. The reasons for this deficit originate in the vary- ing goals of argumentation: persuading audiences, resolving disputes, achieving agreement, complet- ing inquiries, and recommending actions (Tindale, 2007). As a result, diverse quality dimensions play a role, which relate to the logic of arguments, to the style and rhetorical effect of argumentation, or to its contribution to a discussion. Consider the fol- lowing argument against the death penalty:1 Everyone has an inalienable human right to life, …

Content truncated to 500 words