Answering Complex Open-domain Questions Through Iterative Query Generation

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 2590–2602, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 2590 Answering Complex Open-domain Questions Through Iterative Query Generation Peng Qi† Xiaowen Lin∗† Leo Mehr∗† Zijian Wang∗‡ Christopher D. Manning† † Computer Science Department ‡ Symbolic Systems Program Stanford University {pengqi, veralin, leomehr, zijwang, manning}@cs.stanford.edu Abstract It is challenging for current one-step retrieve- and-read question answering (QA) systems to answer questions like “Which novel by the au- thor of ‘Armada’ will be adapted as a feature film by Steven Spielberg?” because the ques- tion seldom contains retrievable clues about the missing entity (here, the author). Answer- ing such a question requires multi-hop reason- ing where one must gather information about the missing entity (or facts) to proceed with further reasoning. We present GOLDEN (Gold Entity) Retriever, which iterates between read- ing context and retrieving more supporting documents to answer open-domain multi-hop questions. Instead of using opaque and com- putationally expensive neural retrieval models, GOLDEN Retriever generates natural language search queries given the question and available context, and leverages off-the-shelf informa- tion retrieval systems to query for missing en- tities. This allows GOLDEN Retriever to scale up efficiently for open-domain multi-hop rea- soning while maintaining interpretability. We evaluate GOLDEN Retriever on the recently proposed open-domain multi-hop QA dataset, HOTPOTQA, and demonstrate that it outper- forms the best previously published model de- spite not using pretrained language models such as BERT. 1 Introduction Open-domain question answering (QA) is an im- portant means for us to make use of knowledge in large text corpora and enables diverse queries without requiring a knowledge schema ahead of time. Enabling such systems to perform multi- step inference can further expand our capability to explore the knowledge in these corpora (e.g., see Figure 1). ∗Equal contribution, order decided by a random number generator. Q: Which novel by the author of “Armada” will be adapted as a feature film by Steven Spielberg? A: Ready Player One Which novel by the author of “Armada” will be adapted as a feature film by Steven Spielberg? The collector The Color Purple (film) Kim Wozencraft novel by the author of “Armada” Armada (novel) Author, Author (novel) Armada Armada author Armada Armada Centre Halley Armada Armada (novel) Armada is a science fiction novel by Ernest Cline, … Ernest Cline Ernest Christy Cline … co-wrote the screenplay for the film adaptation of Ready Player One, directed by Stephen Spielberg. Search Results with queries derived from the original question Figure 1: An example of an open-domain multi-hop question from the HOTPOTQA dev set, where “Ernest Cline” is the missing entity. Note from the search re- sults that it cannot be easily retrieved based on merely the question. (Best viewed in color) Fueled by the recently proposed large-scale QA datasets such as SQuAD (Rajpurkar et al., 2016, 2018) and TriviaQA (Joshi et al., 2017), much progress has been made in open-domain …

Content truncated to 500 words