Repurposing Entailment for Multi-Hop Question Answering Tasks

Proceedings of NAACL-HLT 2019, pages 2948–2958 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2948 Repurposing Entailment for Multi-Hop Question Answering Tasks Harsh Trivedi♣, Heeyoung Kwon♣, Tushar Khot♠, Ashish Sabharwal♠, Niranjan Balasubramanian♣ ♣ Stony Brook University, Stony Brook, U.S.A. {hjtrivedi,heekwon,niranjan}@cs.stonybrook.edu ♠ Allen Institute for Artificial Intelligence, Seattle, U.S.A. {tushark,ashishs}@allenai.org Abstract Question Answering (QA) naturally reduces to an entailment problem, namely, verifying whether some text entails the answer to a ques- tion. However, for multi-hop QA tasks, which require reasoning with multiple sentences, it remains unclear how best to utilize entailment models pre-trained on large scale datasets such as SNLI, which are based on sentence pairs. We introduce Multee, a general architecture that can effectively use entailment models for multi-hop QA tasks. Multee uses (i) a local module that helps locate important sentences, thereby avoiding distracting information, and (ii) a global module that aggregates informa- tion by effectively incorporating importance weights. Importantly, we show that both mod- ules can use entailment functions pre-trained on a large scale NLI datasets. We evaluate per- formance on MultiRC and OpenBookQA, two multihop QA datasets. When using an entail- ment function pre-trained on NLI datasets, Multee outperforms QA models trained only on the target QA datasets and the OpenAI transformer models. 1 Introduction How can we effectively use textual entailment models for question answering? Previous attempts at this have resulted in limited success (Harabagiu and Hickl, 2006; Sacaleanu et al., 2008; Clark et al., 2012). With recent large scale entailment datasets (Bowman et al., 2015; Williams et al., 2018; Khot et al., 2018) pushing entailment mod- els to high accuracies (Chen et al., 2017; Parikh et al., 2016; Wang et al., 2017), we re-visit this challenge and propose a novel method for re- purposing neural entailment models for QA. A key difficulty in using entailment models for QA turns out to be the mismatch between the inputs to the two tasks: large-scale entail- ment datasets are typically framed at a sentence Figure 1: An example illustrating the challenges in us- ing sentence-level entailment model for multi-sentence reasoning needed for QA, and the high-level approach used in Multee. level, whereas question answering requires verify- ing whether multiple sentences, taken together as a premise, entail a hypothesis. There are two straightforward ways to address this mismatch: (1) aggregate independent entail- ment decisions over each premise sentence, or (2) make a single entailment decision after concate- nating all premise sentences. Neither approach is fully satisfactory. To understand why, consider the set of premises in Figure 1, which entail the hypothesis Hc. Specifically, the combined infor- mation in P1 and P3 entails Hc, which corre- sponds to the correct answer Cambridge. On one hand, aggregating independent decisions will fail because no individual premise entails HC. On the other hand, simply concatenating premises to form a single paragraph will fail because distract- ing information in P2 and P4 can muddle use- ful information in P1 and P3. An effective ap- proach, therefore, must recognize relevant …

Content truncated to 500 words