TakeLab at SemEval-2018 Task12: Argument Reasoning Comprehension with Skip-Thought Vectors

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 1133–1136 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics TakeLab at SemEval-2018 Task12: Argument Reasoning Comprehension with Skip-Thought Vectors Ana Brassard, Tin Kuculo, Filip Boltuˇzi´c, Jan ˇSnajder Text Analysis and Knowledge Engineering Lab Faculty of Electrical Engineering and Computing, University of Zagreb Unska 3, 10000 Zagreb, Croatia {name.surname}@fer.hr Abstract This paper describes our system for the SemEval-2018 Task 12: Argument Reasoning Comprehension Task. We utilize skip-thought vectors, sentence-level distributional vectors inspired by the popular word embeddings and the skip-gram model. We encode preprocessed sentences from the dataset into vectors, then perform a binary supervised classification of the warrant that justifies the use of the reason as support for the claim. We explore a few vari- ations of the model, reaching 54.1% accuracy on the test set, which placed us 16th out of 22 teams participating in the task. 1 Introduction Reasoning is the process of thinking in a logical way to form a conclusion. Inferring conclusions us- ing commonsense reasoning has become a popular topic in NLP. Textual entailment (TE) aims to de- termine whether a hypothesis can be inferred from a premise (Dagan et al., 2006). Approaches to solv- ing TE have ranged from robust approaches based on shallow lexical and semantic features (Marelli et al., 2014) to formal computational semantics approaches based on translating sentences into log- ical form (Beltagy et al., 2015). The current state- of-the-art approaches to TE use deep learning for natural logic inference to capture human deductive reasoning (Bowman et al., 2015; Rockt¨aschel et al., 2015). In online discussions, when arguing for or against a stance, people provide arguments leaving their readers to rely on common sense and non- deductive reasoning to evaluate the validity of their arguments. Human annotators can infer the rea- sons from claims fairly well (Boltuˇzi´c and ˇSnajder, 2014; Hasan and Ng, 2014), even when additional (implicit) premises are required to make reasoning deductive (Boltuˇzi´c and ˇSnajder, 2016). Habernal et al. (2018) emphasize the importance of implicit premises in argumentation by introducing the ar- gument reasoning comprehension task, where one chooses between two mutually exclusive warrants to make a reason warrant the claim. They demon- strate that human experts can perform this task extremely well (up to 90% accuracy). In this paper, we describe a system for solving the argument reasoning comprehension task, with which we participated in the SemEval-2018 Task 12. Given the reason R and claim C, debate title, debate description, and two warrants, W1 and W2, the task is to choose warrant W that justifies the use of R as support for C. For all warrant pairs (W1, W2), it holds that if warrant W1 is W, then W2 is ¬W, which justifies the use of R as support for ¬C and vice versa. Our system frames the problem as supervised classification and utilizes skip-thought vectors to represent sentences. Our system (TakeLab) ranked 16th out of 22 systems submitted to the SemEval- 2018 …

Content truncated to 500 words