On Complementarity Objectives for Hybrid Retrieval

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 13357–13368 July 9-14, 2023 ©2023 Association for Computational Linguistics On Complementarity Objectives for Hybrid Retrieval Dohyeon Lee Seoul National University waylight3@snu.ac.kr Seung-won Hwang∗ Seoul National University seungwonh@snu.ac.kr Kyungjae Lee† LG AI Research kyungjae.lee@lgresearch.ai Seungtaek Choi† Riiid seungtaek.choi@riiid.co Sunghyun Park† LG AI Research sunghyun.park@lgresearch.ai Abstract Dense retrieval has shown promising results in various information retrieval tasks, and hybrid retrieval, combined with the strength of sparse retrieval, has also been actively studied. A key challenge in hybrid retrieval is to make sparse and dense complementary to each other. Ex- isting models have focused on dense models to capture “residual” features neglected in the sparse models. Our key distinction is to show how this notion of residual complementarity is limited, and propose a new objective, denoted as RoC (Ratio of Complementarity), which cap- tures a fuller notion of complementarity. We propose a two-level orthogonality designed to improve RoC, then show that the improved RoC of our model, in turn, improves the per- formance of hybrid retrieval. Our method out- performs all state-of-the-art methods on three representative IR benchmarks: MSMARCO- Passage, Natural Questions, and TREC Ro- bust04, with statistical significance. Our find- ing is also consistent in various adversarial set- tings. 1 Introduction Representing and matching queries and documents (or answers) is crucial for designing models for Information Retrieval (IR) and open-domain Ques- tion Answering (QA). Existing approaches have been categorized into sparse and dense retrieval. Classic sparse (or symbolic) retrieval such as BM25 (Robertson and Zaragoza, 2009), quanti- fies the lexical overlaps (or exact matches) be- tween query q and document d, weighted by term frequency (tf) and inverse document frequency (idf). Such computation can be efficiently local- ized to a few high-scoring q-d pairs with an in- verted index, may fail to match pairs with term mismatches. For example, a text pair with identical ∗Corresponding Author †Work done before joining current affiliation. intent—“facebook change password” and “fb mod- ify passwd”—does not share any common word, so the pair cannot be matched by lexical retrieval. To overcome such mismatches, dense retrieval models, such as BERT-based DPR (Karpukhin et al., 2020) or coCondenser (Gao and Callan, 2021), aim to support soft “semantic matching”, by encoding queries and documents into low- dimensional embedding vectors. Dense representa- tion is trained so that “password” and “passwd” are located close in the space even though they have different lexical representations. These complementary advantages of each model have naturally motivated hybrid models (Gao et al., 2020; Yadav et al., 2020; Ma et al., 2021), which we denote as BM25+DPR, extracting scores from both models and selecting documents with the highest linearly combined scores. To illustrate how we advance BM25+DPR base- line, Figure 1(a) shows Recall@10 of BM25+DPR on Natural Questions, where a yellow circle, repre- sents questions answerable by BM25, or S, and a blue circle, represents those answerable by DPR, or D. Desirably, two retrievers together should cover all questions in …

Content truncated to 500 words