Generative Multi-hop Retrieval

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1417 - 1436 December 7-11, 2022 ©2022 Association for Computational Linguistics Generative Multi-hop Retrieval Hyunji Lee Sohee Yang Hanseok Oh Minjoon Seo KAIST AI {hyunji.amy.lee, sohee.yang, hanseok, minjoon}@kaist.ac.kr Abstract A common practice for text retrieval is to use an encoder to map the documents and the query to a common vector space and perform a near- est neighbor search (NNS); multi-hop retrieval also often adopts the same paradigm, usually with a modification of iteratively reformulating the query vector so that it can retrieve different documents at each hop. However, such a bi- encoder approach has limitations in multi-hop settings; (1) the reformulated query gets longer as the number of hops increases, which further tightens the embedding bottleneck of the query vector, and (2) it is prone to error propagation. In this paper, we focus on alleviating these lim- itations in multi-hop settings by formulating the problem in a fully generative way. We pro- pose an encoder-decoder model that performs multi-hop retrieval by simply generating the entire text sequences of the retrieval targets, which means the query and the documents in- teract in the language model’s parametric space rather than L2 or inner product space as in the bi-encoder approach. Our approach, Genera- tive Multi-hop Retrieval (GMR), consistently achieves comparable or higher performance than bi-encoder models in five datasets while demonstrating superior GPU memory and stor- age footprint.1 1 Introduction Finding the relevant knowledge from a massive collection of information is often formulated as a text retrieval problem. A large portion of the text retrieval literature focuses on finding the single most relevant paragraph or document (i.e., no hop) to the given query (Karpukhin et al., 2020; Chen et al., 2017). When we cannot answer a query with a single document, the task is often formulated as a multi-hop retrieval problem, where one needs to retrieve multiple documents that together provide 1https://github.com/amy-hyunji/Generative-Multihop- Retrieval sufficient evidence to answer the query (Yang et al., 2018; Joshi et al., 2017; Dalvi et al., 2021). For example, to answer the question “Where did the form of music played by Die Rhöner Säuwäntzt originate?” (Figure 1), we first need to retrieve the form of music played by Die Rhöner Säuwäntzt and then where the form originated from. No-hop and multi-hop retrieval tasks are often approached by encoding both the query and re- trieval sequences to a common vector space and then finding the sequence whose embedding is clos- est to the query. This bi-encoder approach for re- trieval is often considered as a de facto standard; heavy computations such as extracting the dense embeddings of the items in the corpus can be done offline, and one can search over a large number of items with low latency through the nearest neighbor search (NNS) or maximum inner product search (MIPS) (Lewis et al., 2020; Chen et al., 2020; Wu et al., 2020; Roller et al., 2021). While such a bi- encoder approach performs well …

Content truncated to 500 words