Zero-shot Neural Passage Retrieval via Domain-targeted Synthetic Question Generation

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 1075–1088 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 1075 Zero-shot Neural Passage Retrieval via Domain-targeted Synthetic Question Generation Ji Ma Ivan Korotkov Yinfei Yang Keith Hall Ryan McDonald Google Research {maji,ivankr,yinfeiy,kbhall,ryanmcd}@google.com Abstract A major obstacle to the wide-spread adoption of neural retrieval models is that they require large supervised training sets to surpass tradi- tional term-based techniques, which are con- structed from raw corpora. In this paper, we propose an approach to zero-shot learning for passage retrieval that uses synthetic question generation to close this gap. The question gen- eration system is trained on general domain data, but is applied to documents in the tar- geted domain. This allows us to create arbitrar- ily large, yet noisy, question-passage relevance pairs that are domain specific. Furthermore, when this is coupled with a simple hybrid term- neural model, first-stage retrieval performance can be improved further. Empirically, we show that this is an effective strategy for building neural passage retrieval models in the absence of large training corpora. Depending on the domain, this technique can even approach the accuracy of supervised models. 1 Introduction Recent advances in neural retrieval have led to advancements on several document, passage and knowledge-base benchmarks (Guo et al., 2016; Pang et al., 2016; Hui et al., 2017; Dai et al., 2018; Gillick et al., 2018; Nogueira and Cho, 2019a; MacAvaney et al., 2019; Yang et al., 2019a,b,c). Most neural passage retrieval systems are, in fact, two stages (Zamani et al., 2018; Yilmaz et al., 2019), illustrated in Figure 1. The first is a true retrieval model (aka first-stage retrieval1) that takes a question and retrieves a set of candidate passages from a large collection of documents. This stage itself is rarely a neural model and most commonly is an term-based retrieval model such as BM25 (Robertson et al., 2004; Yang et al., 2017), though there is recent work on neural models (Zamani et al., 2018; Dai and Callan, 2019; Chang et al., 1Also called open domain retrieval. Document Collection Question Retrieval Model Rescoring Model Figure 1: End-to-end neural retrieval. A first-stage model over a large collection returns a smaller set of relevant passages which are reranked by a rescorer. 2020; Karpukhin et al., 2020; Luan et al., 2020). This is usually due to the computational costs re- quired to dynamically score large-scale collections. Another consideration is that BM25 is often high quality (Lin, 2019). After first-stage retrieval, the second stage uses a neural model to rescore the fil- tered set of passages. Since the size of the filtered set is small, this is feasible. The focus of the present work is methods for building neural models for first-stage passage re- trieval for large collections of documents. While rescoring models are key components to any re- trieval system, they are out of the scope of this study. Specifically, we study the zero-shot setting where there is no target-domain supervised training …

Content truncated to 500 words