Context Generation Improves Open Domain Question Answering

Findings of the Association for Computational Linguistics: EACL 2023, pages 793–808 May 2-6, 2023 ©2023 Association for Computational Linguistics Context Generation Improves Open Domain Question Answering Dan Su‡∗, Mostofa Patwary§, Shrimai Prabhumoye§, Peng Xu§, Ryan Prenger§, Mohammad Shoeybi§, Pascale Fung‡, Anima Anandkumar§, Bryan Catanzaro§ ‡The Hong Kong University of Science and Technology, §NVIDIA dsu@connect.ust.hk, mpatwary@nvidia.com Abstract Closed-book question answering (QA) requires a model to directly answer an open-domain question without access to any external knowl- edge. Prior work on closed-book QA either directly finetunes or prompts a pretrained lan- guage model (LM) to leverage the stored knowl- edge. However, they do not fully exploit the parameterized knowledge. To address this inef- ficiency, we propose a two-stage, closed-book QA framework which employs a coarse-to-fine approach to extract the relevant knowledge and answer a question. We first generate a related context for a given question by prompting a pre- trained LM. We then prompt the same LM to generate an answer using the generated context and the question. Additionally, we marginalize over the generated contexts to improve the ac- curacies and reduce context uncertainty. Exper- imental results on three QA benchmarks show that our method significantly outperforms previ- ous closed-book QA methods. For example on TriviaQA, our method improves exact match ac- curacy from 55.3% to 68.6%, and is on par with open-book QA methods (68.6% vs. 68.0%). Our results show that our new methodology is able to better exploit the stored knowledge in pretrained LMs without adding extra learnable parameters or needing finetuning, and paves the way for hybrid models that integrate pretrained LMs with external knowledge. 1 Introduction Open-domain question answering (ODQA) pro- duces an answer to a given question in the form of natural language, and the task has been exten- sively studied in recent years. Significant progress on ODQA has been made by developing the open- book QA methods (Chen et al., 2017; Lewis et al., 2020b; Guu et al., 2020; Izacard and Grave, 2021; ∗ This work was done when the first author was an in- tern at NVIDIA. Corresponding authors: Dan Su, Mostofa Patwary. Generated Conext 1: Richard Marx (songwriter) Richard Marx (born September 16, 1963) is an American adult contemporary and pop/rock singer, songwriter, musician, and record producer. … Three of Marx's singles have reached number one on the U.S. Adult Contemporary chart: "Hold on to the Nights", "Satisfied", and "Right Here Waiting". … Generated Context 3: Kim Wilde Kim Wilde Kim Smith (born Kim Smith on 18 November 1960), better known by her stage name Kim Wilde, is an English pop singer, author, DJ and television personality. She first saw success in 1981 with her debut single, "Kids in America". It reached number two in the UK and was followed by a string of top ten hits. … Generated Conext 2: Richard Marx Richard Allan Marx (born September 16, 1963) is … Marx first came into the spotlight with the release of his debut single, "Don't Mean Nothing", from his eponymous debut album. The …

Content truncated to 500 words