Graph-based Argument Quality Assessment

Proceedings of Recent Advances in Natural Language Processing, pages 1268–1280 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_143 1268 Graph-based Argument Quality Assessment Ekaterina Saveleva and Volha Petukhova and Marius Mosbach and Dietrich Klakow Spoken Language Systems Group, Saarland Informatics Campus Saarland University, Saarbr¨ucken, Germany {esaveleva,v.petukhova,mmosbach,dietrich.klakow}@lsv.uni-saarland.de Abstract The paper presents a novel discourse-based ap- proach to argument quality assessment defined as a graph classification task, where the depth of reasoning (argumentation) is evident from the number and type of detected discourse units and relations between them. We success- fully applied state-of-the-art discourse parsers and machine learning models to reconstruct argument graphs with the identified and clas- sified discourse units as nodes and relations between them as edges. Then Graph Neural Networks were trained to predict the argument quality assessing its acceptability, relevance, sufficiency and overall cogency. The obtained accuracy ranges from 74.5% to 85.0% and in- dicates that discourse-based argument struc- tures reflect qualitative properties of natural language arguments. The results open many interesting prospects for future research in the field of argumentation mining. 1 Introduction Argumentation modelling and mining are steadily gaining attention of the broad natural language pro- cessing and engineering community. In many stud- ies and applications, assessment of the argument quality plays an important role. The ability to con- struct good arguments and engage in argumenta- tive discussions is assessed by argumentation sys- tems focusing on training hypothetical reasoning, creating and structuring arguments (Ashley et al., 2007), preventing opinion manipulation, detecting inconsistent arguments in online discussions and addressing different standpoints, attacking or sup- porting claims with evidence (DebateGraph1 and TruthMapping2) as well as on the use of multi- modal rhetorical devices (Petukhova et al., 2017a). Assessment of argument quality, its organization, 1http://debategraph.org/ 2https://www.truthmapping.com/ clarity, adherence and strength, are approached by several authors as sub-tasks in the evaluation of written essays (Stab and Gurevych, 2014; Persing and Ng, 2015; Wachsmuth et al., 2016; Stab and Gurevych, 2017). Online content is searched to filter or weight the validity of statements and fac- toids (Rowe and Butters, 2009), to identify fake news and false claims (Popat et al., 2018) and to detect opinion manipulation (Cambria et al., 2010). While the acceptability of an argument in the presence of other supporting or attacking argu- ments has been addressed (Dung, 1995; Cayrol and Lagasquie-Schiex, 2005), ‘local’ argument quality still deserves our attention – an argument built on a certain set of conditions, is logically strong, rhetor- ically convincing, socially undistorted by virtue of its intrinsic properties. In this paper, we present a novel approach to assessing the structural strength and inferential weakness of arguments as merits of argument co- gency. The approach relies on the discourse-based reconstruction of argumentation schemes. For this, we apply state-of-the-art discourse parsers and machine learning models to reconstruct argument graphs where the identified discourse units are rep- resented as nodes and the classified discourse re- lations between them as edges. A Graph Neural Network (GNN) model is built to predict the quality (low vs high) of the reconstructed argument graphs …

Content truncated to 500 words