Generation-Augmented Retrieval for Open-Domain Question Answering

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 4089–4100 August 1–6, 2021. ©2021 Association for Computational Linguistics 4089 Generation-Augmented Retrieval for Open-Domain Question Answering Yuning Mao1∗, Pengcheng He2, Xiaodong Liu3, Yelong Shen2, Jianfeng Gao3, Jiawei Han1, Weizhu Chen2 1University of Illinois, Urbana-Champaign 2Microsoft Azure AI 3Microsoft Research 1{yuningm2, hanj}@illinois.edu 2,3{penhe, xiaodl, yeshe, jfgao,wzchen}@microsoft.com Abstract We propose Generation-Augmented Retrieval (GAR) for answering open-domain questions, which augments a query through text genera- tion of heuristically discovered relevant con- texts without external resources as supervi- sion. We demonstrate that the generated con- texts substantially enrich the semantics of the queries and GAR with sparse representations (BM25) achieves comparable or better per- formance than state-of-the-art dense retrieval methods such as DPR (Karpukhin et al., 2020). We show that generating diverse contexts for a query is beneficial as fusing their results con- sistently yields better retrieval accuracy. More- over, as sparse and dense representations are often complementary, GAR can be easily com- bined with DPR to achieve even better per- formance. GAR achieves state-of-the-art per- formance on Natural Questions and TriviaQA datasets under the extractive QA setup when equipped with an extractive reader, and con- sistently outperforms other retrieval methods when the same generative reader is used.1 1 Introduction Open-domain question answering (OpenQA) aims to answer factoid questions without a pre-specified domain and has numerous real-world applications. In OpenQA, a large collection of documents (e.g., Wikipedia) are often used to seek information per- taining to the questions. One of the most com- mon approaches uses a retriever-reader architecture (Chen et al., 2017), which first retrieves a small sub- set of documents using the question as the query and then reads the retrieved documents to extract (or generate) an answer. The retriever is crucial as it is infeasible to examine every piece of information in the entire document collection (e.g., millions of Wikipedia passages) and the retrieval accuracy bounds the performance of the (extractive) reader. ∗Work was done during internship at Microsoft Azure AI. 1Our code is available at https://github.com/ morningmoni/GAR. Early OpenQA systems (Chen et al., 2017) use classic retrieval methods such as TF-IDF and BM25 with sparse representations. Sparse methods are lightweight and efficient, but unable to per- form semantic matching and fail to retrieve rele- vant passages without lexical overlap. More re- cently, methods based on dense representations (Guu et al., 2020; Karpukhin et al., 2020) learn to embed queries and passages into a latent vector space, in which text similarity beyond lexical over- lap can be measured. Dense retrieval methods can retrieve semantically relevant but lexically differ- ent passages and often achieve better performance than sparse methods. However, the dense mod- els are more computationally expensive and suffer from information loss as they condense the entire text sequence into a fixed-size vector that does not guarantee exact matching (Luan et al., 2020). There have been some recent studies on query re- formulation with text generation for other …

Content truncated to 500 words