SNU_IDS at SemEval-2018 Task 12: Sentence Encoder with Contextualized Vectors for Argument Reasoning Comprehension

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 1083–1088 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics SNU IDS at SemEval-2018 Task 12: Sentence Encoder with Contextualized Vectors for Argument Reasoning Comprehension Taeuk Kim, Jihun Choi and Sang-goo Lee Department of Computer Science and Engineering, Seoul National University, Seoul, Korea {taeuk,jhchoi,sglee}@europa.snu.ac.kr Abstract We present a novel neural architecture for the Argument Reasoning Comprehension task of SemEval 2018. It is a simple neural network consisting of three parts, collectively judging whether the logic built on a set of given sen- tences (a claim, reason, and warrant) is plau- sible or not. The model utilizes contextual- ized word vectors pre-trained on large machine translation (MT) datasets as a form of trans- fer learning, which can help to mitigate the lack of training data. Quantitative analysis shows that simply leveraging LSTMs trained on MT datasets outperforms several baselines and non-transferred models, achieving accura- cies of about 70% on the development set and about 60% on the test set. 1 Introduction The Argument Reasoning Comprehension Task (Habernal et al., 2018) is a newly released task that tackles the core of reasoning in natural lan- guage argumentation, highlighting the importance of implicit warrants. Even though the task could be regarded as sim- ple binary classification, it is quite challenging in several perspectives. First, the task requires human-level reasoning to judge whether a claim supported by a reason and a warrant is logically correct. Second, common knowledge, which is not present in the input sentences themselves, is often required to solve the problem. Third, even though each instance of the data is helpful, the number of training data is relatively small to train prevailing complex neural models such as con- volutional neural networks (Kim, 2014; Kalch- brenner et al., 2014) and recurrent neural net- works (Hochreiter and Schmidhuber, 1997; Chung et al., 2014) with (or without) attention mecha- nisms (Liu et al., 2016; Lin et al., 2017). In this paper, we propose a new architec- ture named SECOVARC1(Sentence Encoder with COnextualized Vectors for Argument Reasoning Comprehension) to deal with the complicated task. The main idea behind our model is that transfer learning can be a remedy to resolve the difficulties we face. With experimental results and analysis, we show that the simple neural model enhanced by transferred knowledge can be competitive, com- pared to complex models trained on the given data only. 2 Related Work 2.1 Argument Reasoning Comprehension The argument reasoning comprehension task is a new dataset whose goal is to choose the correct implicit reasoning from two warrants, given a nat- ural language argument with a reason and a claim. It consists of about 2K crowdsourced instances, each of which has a title and a short description of the debate from which the claim, reason, and two candidates arose. For more details, refer to Habernal et al. (2018). 2.2 Transfer Learning in NLP Transfer learning is a classic technique in machine learning, which seeks to transfer beneficial knowl- …

Content truncated to 500 words