Attention-Based Convolutional Neural Network for Machine Comprehension

Proceedings of 2016 NAACL Human-Computer Question Answering Workshop, pages 15–21, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics Attention-Based Convolutional Neural Network for Machine Comprehension Wenpeng Yin, Sebastian Ebert, Hinrich Sch¨utze LMU Munich, Germany wenpeng, ebert@cis.lmu.de Abstract Understanding open-domain text is one of the primary challenges in NLP. Machine com- prehension benchmarks evaluate a system’s ability to understand text based on the text content only. In this work, we investi- gate machine comprehension on MCTest, a question answering (QA) benchmark. Prior work is mainly based on feature engineer- ing approaches. We come up with a neu- ral network framework, named hierarchical attention-based convolutional neural network (HABCNN), to address this task without any manually designed features. Specifically, we explore HABCNN for this task by two routes, one is through traditional joint mod- eling of document, question and answer, one is through textual entailment. HABCNN em- ploys an attention mechanism to detect key phrases, key sentences and key snippets that are relevant to answering the question. Exper- iments show that HABCNN outperforms prior deep learning approaches by a big margin. 1 Introduction Machine comprehension is an open-domain question-answering problem which contains factoid questions, but the answers can be derived by extrac- tion or induction of key clues. Figure 1 shows one example in MCTest (Richardson et al., 2013). Each example consists of one document, four associated questions; each question is followed by four answer candidates of which only one is correct. Questions in MCTest have two categories; “one” questions can be answered based on a single sentence from doc- ument where “multiple” questions require several Figure 1: One example with 2 out of 4 questions in the MCTest. “*” marks correct answer. sentences. To correctly answer the first question in the example, the two blue sentences are required; for the second question instead, we only need the red sentence. The following observations hold for the whole MCTest. (i) Most of the sentences in the document are irrelevant for a given question. It hints that we need to pay attention to just some key regions. (ii) Answer candidates vary in length and abstraction level and usually do not appear in the document. For example, candidate B for the second question is “outside”, which is one word and does not exist in the document, while the answer candidates for the first question are longer texts with some auxiliary words like “Because” in the text. This requires our system to handle flexible texts via extraction as well as abstraction. (iii) Some questions require multiple sentences to infer the answer, and those vital sentences mostly appear close to each other (we call them snippet). Hence, our system should be able to make a choice or compromise between potential single-sentence clue 15 and snippet clue. Prior work is mostly based on feature engineering. We take the lead in presenting a deep neural network without linguistic feature engineering. Concretely, we propose HABCNN, a hierarchical attention-based convolutional neural network, to ad- dress this task in two …

Content truncated to 500 words