Learning Answer-Entailing Structures for Machine Comprehension

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 239–249, Beijing, China, July 26-31, 2015. c⃝2015 Association for Computational Linguistics Learning Answer-Entailing Structures for Machine Comprehension Mrinmaya Sachan1∗ Avinava Dubey1∗ Eric P. Xing1 Matthew Richardson2 1Carnegie Mellon University 2Microsoft Research 1{mrinmays, akdubey, epxing}@cs.cmu.edu 2mattri@microsoft.com Abstract Understanding open-domain text is one of the primary challenges in NLP. Ma- chine comprehension evaluates the sys- tem’s ability to understand text through a series of question-answering tasks on short pieces of text such that the correct answer can be found only in the given text. For this task, we posit that there is a hid- den (latent) structure that explains the rela- tion between the question, correct answer, and text. We call this the answer-entailing structure; given the structure, the correct- ness of the answer is evident. Since the structure is latent, it must be inferred. We present a unified max-margin framework that learns to find these hidden structures (given a corpus of question-answer pairs), and uses what it learns to answer machine comprehension questions on novel texts. We extend this framework to incorporate multi-task learning on the different sub- tasks that are required to perform machine comprehension. Evaluation on a publicly available dataset shows that our frame- work outperforms various IR and neural- network baselines, achieving an overall accuracy of 67.8% (vs. 59.9%, the best previously-published result.) 1 Introduction Developing an ability to understand natural lan- guage is a long-standing goal in NLP and holds the promise of revolutionizing the way in which peo- ple interact with machines and retrieve informa- tion (e.g., for scientific endeavor). To evaluate this ability, Richardson et al. (2013) proposed the task of machine comprehension (MCTest), along with ∗*Work started while the first two authors were interns at Microsoft Research, Redmond. a dataset for evaluation. Machine comprehension evaluates a machine’s understanding by posing a series of reading comprehension questions and as- sociated texts, where the answer to each question can be found only in its associated text. Solutions typically focus on some semantic interpretation of the text, possibly with some form of probabilistic or logical inference, in order to answer the ques- tions. Despite significant recent interest (Burges, 2013; Weston et al., 2014; Weston et al., 2015), the problem remains unsolved. In this paper, we propose an approach for ma- chine comprehension. Our approach learns latent answer-entailing structures that can help us an- swer questions about a text. The answer-entailing structures in our model are closely related to the inference procedure often used in various mod- els for MT (Blunsom and Cohn, 2006), RTE (MacCartney et al., 2008), paraphrase (Yao et al., 2013b), QA (Yih et al., 2013), etc. and correspond to the best (latent) alignment between a hypoth- esis (formed from the question and a candidate answer) with appropriate snippets in the text that are required to answer the question. An example of such an answer-entailing structure is given in Figure 1. The …

Content truncated to 500 words