Lyb3b at SemEval-2018 Task 12: Ensemble-based Deep Learning Models for Argument Reasoning Comprehension Task

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 1137–1141 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics Lyb3b at SemEval-2018 Task 12: Ensemble-based Deep Learning Models for Argument Reasoning Comprehension Task Yongbin Li1,2, Xiaobing Zhou1,∗ 1Yunnan University, Kunming, Yunnan, P.R. China 2Zunyi Medical University, Zunyi, Guizhou, P.R. China ∗ Corresponding author, zhouxb.cn@gmail.com Abstract Reasoning is a crucial part of natural language argumentation. In order to comprehend an ar- gument, we have to reconstruct and analyze its reasoning. In this task, given a natural lan- guage argument with a reason and a claim, the goal is to choose the correct implicit reason- ing from two options, in order to form a rea- sonable structure of (Reason, Warrant, Claim). Our approach is to build distributed word em- bedding of reason, warrant and claim respec- tively, meanwhile, we use a series of frame- works such as CNN model, LSTM model, GRU with attention model and biLSTM with attention model for processing word vector. Finally, ensemble mechanism is used to inte- grate the results of each framework to improve the final accuracy. Experiments demonstrate superior performance of ensemble mechanism compared to each separate framework. We are the 11th in official results, the final model can reach a 0.568 accuracy rate on the test dataset. 1 Introduction Argument reasoning comprehension is a crucial part of natural language argumentation, and the re- alization of argument reasoning requires the un- derstanding of the deep meaning of the text by the computer. At the same time, argument rea- soning is also an important evaluation criterion for the understanding of natural language by com- puter. This paper is based on the argument rea- soning comprehension task proposed by (Haber- nal et al., 2018), which proposed a complex, yet scalable crowdsourcing process, and created a new freely licensed dataset based on authentic argu- ments from news comments. The dataset consists of three parts: train dataset, validation dataset and test dataset, with the quantity being 1210, 316 and 444 respectively. The task is formally defined as follow: given an argument consisting of a reason R and a claim C along with the title and a short description of the debate they occur in, identify the correct war- rant W from two candidates, the goal is to select the correct warrant W that explains reasoning of this particular argument. There are only two op- tions given and only one answer is correct. The key point of the task is that it is difficult to find the answer through the shallow semantics, and the answer is usually implicit. Being a binary classification task, through pre- liminary experiment, our approach is combining debate title and description into reason, and split- ting a sample {R (with debate title and de- scription) ; C; W 0; W 1; correct label} into two quadruples, which are {R; C; W 0; label} and {R; C; W 1; label}. On the validation, we employ the same processing mode, determin- ing the matching degree …

Content truncated to 500 words