Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 874–880 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 874 Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering Gautier Izacard1,2,3 Edouard Grave1 1 Facebook AI Research, Paris 2 ENS, PSL University, Paris 3 Inria, Paris gizacard|egrave@fb.com Abstract Generative models for open domain question answering have proven to be competitive, with- out resorting to external knowledge. While promising, this approach requires to use mod- els with billions of parameters, which are ex- pensive to train and query. In this paper, we investigate how much these models can ben- efit from retrieving text passages, potentially containing evidence. We obtain state-of-the- art results on the Natural Questions and Triv- iaQA open benchmarks. Interestingly, we ob- serve that the performance of this method sig- nificantly improves when increasing the num- ber of retrieved passages. This is evidence that sequence-to-sequence models offers a flexible framework to efficiently aggregate and com- bine evidence from multiple passages. 1 Introduction Recently, several works have shown that factual information can be extracted from large scale language models trained on vast quantities of data (Radford et al., 2019; Petroni et al., 2019; Jiang et al., 2019; Talmor et al., 2019). Building on that observation and the advances in pretrain- ing of natural language processing models, Roberts et al. (2020) introduced a generative model for open domain question answering. Without relying on external knowledge, this method obtained compet- itive results on several benchmarks. However, it requires models containing billions of parameters, since all the information needs to be stored in the weights. This makes models expensive to query and train. In this paper, we investigate how much this method could benefit from having access to an external source of knowledge, such as Wikipedia. Retrieval based approaches were previously con- sidered in the context of open domain question answering with extractive models (Chen et al., 2017). In that case, systems start by retrieving Alan Turing was a British computer scientist. Born in Maida Vale, London… Where was Alan Turing born? Generative seq2seq model Maida Vale, London Figure 1: A simple approach to open domain question answering. First, it retrieves support text passages from an external source of knowledge such as Wikipedia. Then, a generative encoder-decoder model produces the answer, conditioned on the question and the re- trieved passages. This approach scales well with the number of retrieved passages, as the performance keeps improving when retrieving up to one hundred passages. support documents, before extracting the answer from these documents. Different retrieval tech- niques have been considered, either using sparse representations based on TF/IDF or using dense embeddings (Guu et al., 2020; Karpukhin et al., 2020). The models which extract the answers are often based on contextualized word representations such as ELMo or BERT (Peters et al., 2018; De- vlin et al., 2019), and predict a span as answer. Aggregating and combining evidence from mul- tiple passages is not straightforward when using …

Content truncated to 500 words