PullNet: Open Domain Question Answering with Iterative Retrieval on Knowledge Bases and Text

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 2380–2390, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 2380 PullNet: Open Domain Question Answering with Iterative Retrieval on Knowledge Bases and Text Haitian Sun Tania Bedrax-Weiss Google ResearchWilliam W. Cohen {haitiansun,tbedrax,wcohen}@google.com Abstract We consider open-domain question answering (QA) where answers are drawn from either a corpus, a knowledge base (KB), or a combina- tion of both of these. We focus on a setting in which a corpus is supplemented with a large but incomplete KB, and on questions that re- quire non-trivial (e.g., “multi-hop”) reasoning. We describe PullNet, an integrated framework for (1) learning what to retrieve and (2) rea- soning with this heterogeneous information to find the best answer. PullNet uses an iterative process to construct a question-specific sub- graph that contains information relevant to the question. In each iteration, a graph convolu- tional network (graph CNN) is used to identify subgraph nodes that should be expanded using retrieval (or “pull”) operations on the corpus and/or KB. After the subgraph is complete, an- other graph CNN is used to extract the answer from the subgraph. This retrieve-and-reason process allows us to answer multi-hop ques- tions using large KBs and corpora. PullNet is weakly supervised, requiring question-answer pairs but not gold inference paths. Experimen- tally PullNet improves over the prior state-of- the art, and in the setting where a corpus is used with incomplete KB these improvements are often dramatic. PullNet is also often supe- rior to prior systems in a KB-only setting or a text-only setting. 1 Introduction Open domain Question Answering (QA) is the task of finding answers to questions posed in nat- ural language, usually using text from a corpus (Dhingra et al., 2017; Joshi et al., 2017; Dunn et al., 2017), or triples from a knowledge base (KB) (Zelle and Mooney, 1996; Zettlemoyer and Collins, 2005; Yih et al., 2015). Both of these ap- proaches have limitations. Even the largest KBs are incomplete (Min et al., 2013), which limits recall of a KB-based QA system. On the other hand, while a large corpus may contain more an- swers than a KB, the diversity of natural language makes corpus-based QA difficult (Chen et al., 2017; Welbl et al., 2018; Kwiatkowski et al., 2019; Yang et al., 2018). In this paper we follow previous research (Sawant et al., 2019; Sun et al., 2018) in deriving answers using both a corpus and a KB. We focus on tasks in which questions require compositional (sometimes called “multi-hop”) reasoning, and a setting in which the KB is incomplete, and hence must be supplemented with information extracted from text. We also restrict ourselves in this paper to answers which correspond to KB entities. For this setting, we propose an integrated framework for (1) learning what to retrieve, from either a cor- pus, a KB, or a combination, and (2) combining this heterogeneous information into a …

Content truncated to 500 words