Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1597–1610 May 2-6, 2023 ©2023 Association for Computational Linguistics Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering Wenhu Chen, Pat Verga, Michiel de Jong†, John Wieting, William W. Cohen Google Research, University of Southern California† {wenhuchen,patverga,jwieting,wcohen}@google.com, msdejong@usc.edu Abstract Existing state-of-the-art methods for open- domain question-answering (ODQA) use an open book approach in which information is first retrieved from a large text corpus or knowledge base (KB) and then reasoned over to produce an answer. A recent alternative is to retrieve from a collection of previously- generated question-answer pairs; this has sev- eral practical advantages including being more memory and compute-efficient. Question- answer pairs are also appealing in that they can be viewed as an intermediate between text and KB triples: like KB triples, they often con- cisely express a single relationship, but like text, have much higher coverage than tradi- tional KBs. In this work, we describe a new QA system that augments a text-to-text model with a large memory of question-answer pairs, and a new pre-training task for the latent step of question retrieval. The pre-training task substantially simplifies training and greatly improves performance on smaller QA bench- marks. Unlike prior systems of this sort, our QA system can also answer multi-hop ques- tions that do not explicitly appear in the col- lection of stored question-answer pairs. 1 Introduction Open-domain question answering (ODQA) is a well-studied knowledge-intensive task. State-of- the-art methods require retrieving relevant knowl- edge from a large corpus or datastore before rea- soning over this retrieved evidence. Most existing methods retrieve documents (Chen et al., 2017a; Lee et al., 2019; Karpukhin et al., 2020) or struc- tured KB triples (Verga et al., 2021). Recently, a few works have proposed retrieving from a collec- tion of question-answer (QA) pairs—an approach made feasible by advances in scalable automatic question generation. In this setting, a new ques- tion is answered by retrieving paraphrases from a question index, and returning the associated answer (Xiao et al., 2021; Lewis et al., 2021). Notably, the RePAQ system from (Lewis et al., 2021) won the 2020 EfficientQA competition (Min et al., 2021), outperforming closed-book QA (CBQA) models by a significant margin and matching the prior SoTA performance on NQ (Kwiatkowski et al., 2019). A collection of QA pairs is appealing for several reasons. As opposed to text passages and much like a KB triple, QA pairs are often concise, tending to express a single relationship. However, unlike KB triples, QA collections have good coverage of actually asked questions like those in standard open QA datasets. RePAQ demonstrated several advan- tageous properties such as memory and computa- tional efficiency, strong selective QA performance (i.e. selectively abstaining from answering), and ef- fective ensembling with text-retrieval QA systems. However, question-retrieval QA systems have several limitations as well. First, there is no large- scale supervised data for question-question re- trieval. This contrasts with the step of retriev- ing text given …

Content truncated to 500 words