Classifying Argumentative Relations Using Logical Mechanisms and Argumentation Schemes

Classifying Argumentative Relations Using Logical Mechanisms and Argumentation Schemes Yohan Jo1 Seojin Bang1 Chris Reed2 Eduard Hovy1 1School of Computer Science, Carnegie Mellon University, United States 2Centre for Argument Technology, University of Dundee, United Kingdom 1{yohanj,seojinb,ehovy}@andrew.cmu.edu, 2c.a.reed@dundee.ac.kr Abstract While argument mining has achieved sig- nificant success in classifying argumentative relations between statements (support, attack, and neutral), we have a limited computa- tional understanding of logical mechanisms that constitute those relations. Most recent studies rely on black-box models, which are not as linguistically insightful as desired. On the other hand, earlier studies use rather simple lexical features, missing logical rela- tions between statements. To overcome these limitations, our work classifies argumenta- tive relations based on four logical and theory-informed mechanisms between two statements, namely, (i) factual consistency, (ii) sentiment coherence, (iii) causal relation, and (iv) normative relation. We demonstrate that our operationalization of these logical mechanisms classifies argumentative relations without directly training on data labeled with the relations, significantly better than several unsupervised baselines. We further demon- strate that these mechanisms also improve supervised classifiers through representation learning. 1 Introduction There have been great advances in argument mining—classifying the argumentative relation between statements as support, attack, or neutral. Recent research has focused on training complex neural networks on large labeled data. However, the behavior of such models remains obscure, and recent studies found evidence that those models may rely on spurious statistics of training data (Niven and Kao, 2019) and superficial cues irrelevant to the meaning of statements, such as discourse markers (Opitz and Frank, 2019). Hence, in this work, we turn to an interpretable method to investigate logical relations between statements, such as causal relations and factual contradiction. Such relations have been underemphasized in earlier studies (Feng and Hirst, 2011; Lawrence and Reed, 2016), possibly because their operationalization was unreliable then. Now that computational semantics is fast developing, our work takes a first step to computationally investigate how logical mechanisms contribute to building argumentative relations between statements and to classification accuracy with and without training on labeled data. To investigate what logical mechanisms gov- ern argumentative relations, we hypothesize that governing mechanisms should be able to classify the relations without directly training on relation- labeled data. Thus, we first compile a set of rules specifying logical and theory-informed mecha- nisms that signal the support and attack relations (§3). The rules are grouped into four mechanisms: factual consistency, sentiment coherence, causal relation, and normative relation. These rules are combined via probabilistic soft logic (PSL) (Bach et al., 2017) to estimate the optimal argumentative relations between statements. We operationalize each mechanism by training seman- tic modules on public datasets so that the modules reflect real-world knowledge necessary for rea- soning (§4). For normative relation, we build a necessary dataset via rich annotation of the normative argumentation schemes argument from consequences and practical reasoning (Walton et al., 2008), by developing a novel and reliable annotation protocol (§5). Our evaluation is based on arguments from kialo.com and debatepedia.org. We first demon- strate that the …

Content truncated to 500 words