Recognizing Insufficiently Supported Arguments in Argumentative Essays

Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 980–990, Valencia, Spain, April 3-7, 2017. c⃝2017 Association for Computational Linguistics Recognizing Insufficiently Supported Arguments in Argumentative Essays Christian Stab† and Iryna Gurevych†‡ †Ubiquitous Knowledge Processing Lab (UKP-TUDA) Department of Computer Science, Technische Universit¨at Darmstadt ‡Ubiquitous Knowledge Processing Lab (UKP-DIPF) German Institute for Educational Research www.ukp.tu-darmstadt.de Abstract In this paper, we propose a new task for as- sessing the quality of natural language ar- guments. The premises of a well-reasoned argument should provide enough evidence for accepting or rejecting its claim. Al- though this criterion, known as sufficiency, is widely adopted in argumentation theory, there are no empirical studies on its appli- cability to real arguments. In this work, we show that human annotators substan- tially agree on the sufficiency criterion and introduce a novel annotated corpus. Fur- thermore, we experiment with feature-rich SVMs and convolutional neural networks and achieve 84% accuracy for automati- cally identifying insufficiently supported arguments. The final corpus as well as the annotation guideline are freely avail- able for encouraging future research on ar- gument quality.1 1 Introduction Argumentation is an omnipresent routine and an integral part of our daily verbal communication. It is a verbal activity that aims at increasing or de- creasing the plausibility of a controversial stand- point (van Eemeren et al., 1996, p. 5). Well- reasoned arguments of high quality are not only important for making thoughtful decisions and persuading a particular audience but also play a major role for drawing widely accepted conclu- sions. Computational argumentation is a recent re- search field in natural language processing that fo- cuses on the analysis of arguments in natural lan- guage texts. Novel advances have a broad applica- 1https://www.ukp.tu-darmstadt.de/data/ argumentation-mining tion potential in various areas like debating tech- nologies (Levy et al., 2014; Rinott et al., 2015), policy making (Sardianos et al., 2015), informa- tion retrieval (Carstens and Toni, 2015), and le- gal decision support (Mochales-Palau and Moens, 2009). Recently, computational argumentation is receiving increasing interest in intelligent writ- ing assistance (Song et al., 2014; Stab et al., 2014) since it enables argumentative writing sup- port systems that provide tailored feedback about arguments in student essays. Most of the existing approaches in computa- tional argumentation consider argumentation as discourse structures and focus on the identifica- tion of arguments in natural language texts. For instance, existing approaches classify text units as argumentative or non-argumentative (Moens et al., 2007), recognize argument components such as claims or premises at the sentence-level (Mochales-Palau and Moens, 2009; Kwon et al., 2007; Eckle-Kohler et al., 2015) or clause-level (Levy et al., 2014; Sardianos et al., 2015), or iden- tify argument structures by classifying pairs of ar- gument components (Stab and Gurevych, 2014). However, these approaches are of limited use for argumentative writing support systems since they do not recognize the weak points of arguments. Despite the comprehensive theoretical frame- work on argument quality in logic and argumen- tation theory (van …

Content truncated to 500 words