What’s Missing: A Knowledge Gap Guided Approach for Multi-hop Question Answering

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 2814–2828, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 2814 What’s Missing: A Knowledge Gap Guided Approach for Multi-hop Question Answering Tushar Khot and Ashish Sabharwal and Peter Clark Allen Institute for Artificial Intelligence, Seattle, WA, U.S.A. {tushark,ashishs,peterc}@allenai.org Abstract Multi-hop textual question answering requires combining information from multiple sen- tences. We focus on a natural setting where, unlike typical reading comprehension, only partial information is provided with each question. The model must retrieve and use additional knowledge to correctly answer the question. To tackle this challenge, we de- velop a novel approach that explicitly identi- fies the knowledge gap between a key span in the provided knowledge and the answer choices. The model, GapQA, learns to fill this gap by determining the relationship between the span and an answer choice, based on re- trieved knowledge targeting this gap. We pro- pose jointly training a model to simultaneously fill this knowledge gap and compose it with the provided partial knowledge. On the Open- BookQA dataset, given partial knowledge, ex- plicitly identifying what’s missing substan- tially outperforms previous approaches. 1 Introduction Reading Comprehension datasets (Richardson et al., 2013; Rajpurkar et al., 2016; Joshi et al., 2017) have gained interest as benchmarks to eval- uate a system’s ability to understand a document via question answering (QA). Since many of these early datasets only required a system to understand a single sentence, new datasets were specifically designed to focus on the problem of multi-hop QA, i.e., reasoning across sentences (Khashabi et al., 2018; Welbl et al., 2018; Yang et al., 2018). While this led to improved language under- standing, the tasks still assume that a system is provided with all knowledge necessary to answer the question. In practice, however, we often only have access to partial knowledge when dealing with such multi-hop questions, and must retrieve additional facts (the knowledge “gaps”) based on Question: Which of these would let the most heat travel through? A) a new pair of jeans. B) a steel spoon in a cafeteria. C) a cotton candy at a store. D) a calvin klein cotton hat. Core Fact: Metal lets heat travel through. Knowledge Gap (similar gaps for other choices): steel spoon in a cafeteria metal. Filled Gap (relation identified using KB): steel spoon in a cafeteria is made of metal. Figure 1: A sample OpenBookQA question, the identi- fied knowledge gap based on partial information in the core fact, and relation (is made of) identified from a KB to fill that gap. the question and the provided knowledge. Our goal is to identify such gaps and fill them using an external knowledge source. The recently introduced challenge of open book question answering (Mihaylov et al., 2018) high- lights this phenomenon. The questions in the corresponding dataset, OpenBookQA, are derived from a science fact in an “open book” of about 1300 facts. To answer …

Content truncated to 500 words