The Argument Reasoning Comprehension Task: Identification and Reconstruction of Implicit Warrants

Proceedings of NAACL-HLT 2018, pages 1930–1940 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics The Argument Reasoning Comprehension Task: Identification and Reconstruction of Implicit Warrants Ivan Habernal† Henning Wachsmuth‡ Iryna Gurevych† Benno Stein‡ † Ubiquitous Knowledge Processing Lab (UKP) and Research Training Group AIPHES Department of Computer Science, Technische Universit¨at Darmstadt, Germany www.ukp.tu-darmstadt.de www.aiphes.tu-darmstadt.de ‡ Faculty of Media, Bauhaus-Universit¨at Weimar, Germany <firstname>.<lastname>@uni-weimar.de Abstract Reasoning is a crucial part of natural lan- guage argumentation. To comprehend an argu- ment, one must analyze its warrant, which ex- plains why its claim follows from its premises. As arguments are highly contextualized, war- rants are usually presupposed and left implicit. Thus, the comprehension does not only require language understanding and logic skills, but also depends on common sense. In this pa- per we develop a methodology for reconstruct- ing warrants systematically. We operationalize it in a scalable crowdsourcing process, result- ing in a freely licensed dataset with warrants for 2k authentic arguments from news com- ments.1 On this basis, we present a new chal- lenging task, the argument reasoning compre- hension task. Given an argument with a claim and a premise, the goal is to choose the cor- rect implicit warrant from two options. Both warrants are plausible and lexically close, but lead to contradicting claims. A solution to this task will define a substantial step towards auto- matic warrant reconstruction. However, exper- iments with several neural attention and lan- guage models reveal that current approaches do not suffice. 1 Introduction Most house cats face enemies. Russia has the op- posite objectives of the US. There is much innova- tion in 3-d printing and it is sustainable. What do the three propositions have in com- mon? They were never uttered but solely presup- posed in arguments made by the participants of online discussions. Presuppositions are a funda- mental pragmatic instrument of natural language argumentation in which parts of arguments are left unstated. This phenomenon is also referred to as 1Available at https://github.com/UKPLab/ argumentreasoning-comprehension-task/, including source codes and supplementary materials. common knowledge (Macagno and Walton, 2014, p. 218), enthymemes (Walton, 2007b, p. 12), tacit major premises (Amossy, 2009, p. 319), or implicit warrants (Newman and Marshall, 1991, p. 8). Wilson and Sperber (2004) suggest that, when we comprehend arguments, we reconstruct their warrants driven by the cognitive principle of relevance. In other words, we go straight for the interpretation that seems most relevant and logi- cal within the given context (Hobbs et al., 1993). Although any incomplete argument can be com- pleted in different ways (Plumer, 2016), it is as- sumed that certain knowledge is shared between the arguing parties (Macagno and Walton, 2014, p. 180). Filling the gap between the claim and premises (aka reasons) of a natural language argument em- pirically remains an open issue, due to the inher- ent difficulty of reconstructing the world knowl- edge and reasoning patterns in arguments. In a di- rect fashion, Boltuˇzi´c and ˇSnajder (2016) let an- notators write down implicit warrants, …

Content truncated to 500 words