An Empirical Evaluation of various Deep Learning Architectures for Bi-Sequence Classification Tasks

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2762–2773, Osaka, Japan, December 11-17 2016. An Empirical Evaluation of various Deep Learning Architectures for Bi-Sequence Classification Tasks Anirban Laha IBM Research India anirlaha@in.ibm.com Vikas Raykar IBM Research India viraykar@in.ibm.com Abstract Several tasks in argumentation mining and debating, question-answering, and natural language inference involve classifying a sequence in the context of another sequence (referred as bi- sequence classification). For several single sequence classification tasks, the current state-of- the-art approaches are based on recurrent and convolutional neural networks. On the other hand, for bi-sequence classification problems, there is not much understanding as to the best deep learn- ing architecture. In this paper, we attempt to get an understanding of this category of problems by extensive empirical evaluation of 19 different deep learning architectures (specifically on dif- ferent ways of handling context) for various problems originating in natural language processing like debating, textual entailment and question-answering. Following the empirical evaluation, we offer our insights and conclusions regarding the architectures we have considered. We also establish the first deep learning baselines for three argumentation mining tasks. 1 Introduction Argumentation mining is a relatively new challenge in corpus-based discourse analysis that involves automatically identifying argumentative structures within a corpus. Many tasks in argumentation min- ing (Lippi and Torroni, 2015a) and debating technologies (Slonim et al., 2014) involve categorizing a sequence in the context of another sequence. For example, in context dependent claim detection (Levy et al., 2014), given a sentence, one task is to identify whether the sentence contains a claim relevant to a particular debatable topic (generally given as a context sentence). Similarly in context dependent evi- dence detection (Rinott et al., 2015), given a sequence (possibly multiple sentences), one task is to detect if the sequence contains an evidence relevant to a particular topic. We refer to such class of problems in computational argumentation as bi-sequence classification problems—given two sequences s and c we want to predict the label for the target sequence s in the context of another sequence c1. Apart from the debating tasks, several other natural language inference tasks fall under the same paradigm of having a pair of sequences. For example, recognizing textual entailment (Bowman et al., 2015), where the task is to predict if the meaning of a sentence can be inferred from the meaning of another sentence. Another class of problems originated from question-answering systems also known as answer selection, where given a question, a candidate answer needs to be classified as an answer to the question at hand or not. Recently, deep learning approaches have obtained very high performance across many different natural language processing tasks. These models can often be trained in an end-to-end fashion and do not require traditional, task-specific feature engineering. For many single sequence classification tasks, the state-of- the-art approaches are based on recurrent neural networks (RNN variants like Long Short-Term Memory (LSTM) (Hochreiter and Schmidhuber, 1997) and Gated Recurrent Unit (GRU) (Cho et al., 2014)) and …

Content truncated to 500 words