ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3715 - 3734 July 10-15, 2022 ©2022 Association for Computational Linguistics ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction Keshav Santhanam∗ Stanford University Omar Khattab∗ Stanford University Jon Saad-Falcon Georgia Institute of Technology Christopher Potts Stanford University Matei Zaharia Stanford University Abstract Neural information retrieval (IR) has greatly advanced search and other knowledge- intensive language tasks. While many neural IR methods encode queries and documents into single-vector representations, late interaction models produce multi-vector repre- sentations at the granularity of each token and decompose relevance modeling into scalable token-level computations. This decomposition has been shown to make late interaction more effective, but it inflates the space footprint of these models by an order of magnitude. In this work, we introduce ColBERTv2, a retriever that couples an aggressive residual compres- sion mechanism with a denoised supervision strategy to simultaneously improve the quality and space footprint of late interaction. We evaluate ColBERTv2 across a wide range of benchmarks, establishing state-of-the-art quality within and outside the training domain while reducing the space footprint of late interaction models by 6–10×. 1 Introduction Neural information retrieval (IR) has quickly domi- nated the search landscape over the past 2–3 years, dramatically advancing not only passage and doc- ument search (Nogueira and Cho, 2019) but also many knowledge-intensive NLP tasks like open- domain question answering (Guu et al., 2020), multi-hop claim verification (Khattab et al., 2021a), and open-ended generation (Paranjape et al., 2022). Many neural IR methods follow a single-vector similarity paradigm: a pretrained language model is used to encode each query and each document into a single high-dimensional vector, and rele- vance is modeled as a simple dot product between both vectors. An alternative is late interaction, in- troduced in ColBERT (Khattab and Zaharia, 2020), where queries and documents are encoded at a finer- granularity into multi-vector representations, and ∗Equal contribution. relevance is estimated using rich yet scalable in- teractions between these two sets of vectors. Col- BERT produces an embedding for every token in the query (and document) and models relevance as the sum of maximum similarities between each query vector and all vectors in the document. By decomposing relevance modeling into token- level computations, late interaction aims to reduce the burden on the encoder: whereas single-vector models must capture complex query–document re- lationships within one dot product, late interaction encodes meaning at the level of tokens and del- egates query–document matching to the interac- tion mechanism. This added expressivity comes at a cost: existing late interaction systems impose an order-of-magnitude larger space footprint than single-vector models, as they must store billions of small vectors for Web-scale collections. Con- sidering this challenge, it might seem more fruit- ful to focus instead on addressing the fragility of single-vector models (Menon et al., 2022) by in- troducing new supervision paradigms for negative mining (Xiong et al., 2020), pretraining (Gao and Callan, 2021), and distillation (Qu et al., …

Content truncated to 500 words