Simple and Effective Unsupervised Redundancy Elimination to Compress Dense Vectors for Passage Retrieval

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2854–2859 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 2854 Simple and Effective Unsupervised Redundancy Elimination to Compress Dense Vectors for Passage Retrieval Xueguang Ma∗, Minghan Li∗, Kai Sun, Ji Xin, and Jimmy Lin David R. Cheriton School of Computer Science University of Waterloo {x93ma, m692li, k49sun, ji.xin, jimmylin}@uwaterloo.ca Abstract Recent work has shown that dense passage re- trieval techniques achieve better ranking accu- racy in open-domain question answering com- pared to sparse retrieval techniques such as BM25, but at the cost of large space and mem- ory requirements. In this paper, we analyze the redundancy present in encoded dense vec- tors and show that the default dimension of 768 is unnecessarily large. To improve space efficiency, we propose a simple unsupervised compression pipeline that consists of princi- pal component analysis (PCA), product quan- tization, and hybrid search. We further inves- tigate other supervised baselines and find sur- prisingly that unsupervised PCA outperforms them in some settings. We perform exten- sive experiments on five question answering datasets and demonstrate that our best pipeline achieves good accuracy–space trade-offs, for example, 48× compression with less than 3% drop in top-100 retrieval accuracy on aver- age or 96× compression with less than 4% drop. Code and data are available at http: //pyserini.io/. 1 Introduction Dense passage retrieval (DPR; Karpukhin et al., 2020) improves end-to-end retrieval accuracy in open-domain question answering (QA) by representing queries and documents in a low- dimensional, dense vector space. However, the vastly increased space and memory demands for storing and loading the dense vectors call for ef- fective compression methods (Izacard et al., 2020). For example, the size of the original DPR vector (flat) index on the Wikipedia corpus is about 61 GB, while its sparse counterpart—BM25 inverted index—only uses 2.4 GB. The staggering increase of around 25× in space requirements only yields an average gain of 2.5% in top-100 accuracy across ∗ Equal contribution five datasets (Ma et al., 2021), indicating potential redundancy in the dense representations. In this work, we quantify redundancy within the dense vectors encoded by the DPR model using explained variance ratio and mutual information. Figure 1 shows that the original 768 dimensions is unnecessarily large as both the mutual informa- tion and explained variance ratio start to plateau at around 256 dimensions. Based on this observation, we further propose a simple yet effective pipeline for dense retrieval compression that includes prin- cipal component analysis (PCA), product quantiza- tion (PQ), and hybrid search to reduce index size while retaining effectiveness. We also compare other compression options, including supervised dimensionality reduction, where we fine-tune a linear projection layer on top of the pre-trained DPR model using relevance labels. Surprisingly, we find that PCA achieves top- 100 retrieval accuracy that is better than the two supervised dimensionality reduction techniques for 256 and 128 dimensions, while supervised techniques outperform (unsupervised) PCA for 64 dimensions. Our techniques support different accuracy–space trade-offs, but one sweet spot …

Content truncated to 500 words