Recovering Gold from Black Sand: Multilingual Dense Passage Retrieval with Hard and False Negative Samples

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10659 - 10670 December 7-11, 2022 ©2022 Association for Computational Linguistics Recovering Gold from Black Sand: Multilingual Dense Passage Retrieval with Hard and False Negative Samples Tianhao Shen1∗, Mingtong Liu2, Ming Zhou2, Deyi Xiong1† 1 College of Intelligence and Computing, Tianjin University, Tianjin, China 2Beijing Lanzhou Technology Co., Ltd., Beijing, China {thshen,dyxiong}@tju.edu.cn, {liumingtong,zhouming}@langboat.com Abstract Negative samples have not been efficiently ex- plored in multilingual dense passage retrieval. In this paper, we propose a novel multilingual dense passage retrieval framework, mHFN, to recover and utilize hard and false negative sam- ples. mHFN consists of three key components: 1) a multilingual hard negative sample aug- mentation module that allows knowledge of indistinguishable passages to be shared across multiple languages and synthesizes new hard negative samples by interpolating representa- tions of queries and existing hard negative sam- ples, 2) a multilingual negative sample cache queue that stores negative samples from pre- vious batches in each language to increase the number of multilingual negative samples used in training beyond the batch size limit, and 3) a lightweight adaptive false negative sample filter that uses generated pseudo labels to separate unlabeled false negative samples and converts them into positive passages in training. We evaluate mHFN on Mr. TyDi, a high-quality multilingual dense passage re- trieval dataset covering eleven typologically di- verse languages, and experimental results show that mHFN outperforms strong sparse, dense and hybrid baselines and achieves new state- of-the-art performance on all languages. Our source code is available at https://github. com/Magnetic2014/mHFN. 1 Introduction Passage retrieval, which matches relevant passages to queries, is an important task in Information Re- trieval (IR). It can be also integrated as a core com- ponent to solve many Natural Language Processing (NLP) problems, e.g., open domain question an- swering (Chen et al., 2017), fact checking (Thorne et al., 2018), etc. Powered by large scale pretrained language models (e.g. BERT (Devlin et al., 2019), ∗Work done as an intern at Beijing Lanzhou Technology Co., Ltd., Beijing, China. †Corresponding author. RoBERTa (Liu et al., 2019), and T5 (Raffel et al., 2020)), dense passage retrieval, which explores dense vector representations to match relevant pas- sages, has attracted growing interest (Gao et al., 2020; Khattab and Zaharia, 2020; Karpukhin et al., 2020; MacAvaney et al., 2020a; Qu et al., 2021; Zhan et al., 2021; Xu et al., 2022; Gao et al., 2021a). Dense passage retrieval models usually adopt a bi-encoder (also known as dual-encoder) architec- ture, one encoder for encoding queries, the other for learning passage representations, which can be computed offline. With dense representations of a query and document, passage ranking is recast as a nearest neighbor search problem that can be effi- ciently solved by similarity search toolkits tailored for dense vectors, such as Faiss (Johnson et al., 2019). Dense passage retrieval models are usually trained via Contrastive Learning (CL), which en- courages query representations to be close to pos- itive (i.e., relevant) passages and away from neg- …

Content truncated to 500 words