Cross-Lingual Phrase Retrieval

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 4193 - 4204 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Cross-Lingual Phrase Retrieval Heqi Zheng12∗, Xiao Zhang1∗, Zewen Chi1∗, Heyan Huang12 Tan Yan1, Tian Lan1, Wei Wei3, Xian-Ling Mao1† 1School of Computer Science and Technology, Beijing Institute of Technology 2Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications 3Huazhong University of Science and Technology {hikiz,yotta,czw,hhy63,yt,lt,maoxl}@bit.edu.cn,weiw@hust.edu.cn Abstract Cross-lingual retrieval aims to retrieve relevant text across languages. Current methods typi- cally achieve cross-lingual retrieval by learn- ing language-agnostic text representations in word or sentence level. However, how to learn phrase representations for cross-lingual phrase retrieval is still an open problem. In this pa- per, we propose XPR, a cross-lingual phrase retriever that extracts phrase representations from unlabeled example sentences. Moreover, we create a large-scale cross-lingual phrase re- trieval dataset, which contains 65K bilingual phrase pairs and 4.2M example sentences in 8 English-centric language pairs. Experimen- tal results show that XPR outperforms state- of-the-art baselines which utilize word-level or sentence-level representations. XPR also shows impressive zero-shot transferability that enables the model to perform retrieval in an unseen language pair during training. Our dataset, code, and trained models are publicly available at github.com/cwszz/XPR/. 1 Introduction Phrase retrieval aims to retrieve relevant phrases from a large phrase set, which is a critical part of information retrieval. Recent studies on phrase re- trieval learn dense representations of phrases, and achieve promising results in entity linking, slot filling, and open-domain question answering tasks (Gillick et al., 2019; Lee et al., 2021a,b). Nonethe- less, most of the studies focus on monolingual sce- narios, leaving the cross-lingual phrase retrieval unexplored. Various methods have been proposed to perform cross-lingual text retrieval, which learns cross- lingual word or sentence representations shared ∗Co-first authors with equal contributions. †Corresponding author. across languages. Cross-lingual word representa- tion methods typically train word embeddings on each language separately, and then learn an em- bedding mapping between the embedding spaces of different languages (Mikolov et al., 2013; Dinu et al., 2014). Then, the bilingual word pairs can be retrieved between vocabularies using nearest neighbor search, which is also known as bilingual lexicon induction (Artetxe et al., 2018; Lample et al., 2018). Cross-lingual sentence retrieval is typically achieved by learning a sentence encoder on multilingual text corpora with self-supervised pretraining tasks (Conneau and Lample, 2019; Conneau et al., 2020), or large-scale parallel cor- pora (Artetxe and Schwenk, 2019), or both (Chi et al., 2021b). The trained sentence encoders pro- duce language-agnostic sentence representations, which enables sentences to be retrieved across lan- guages. Despite the effectiveness of word-level and sentence-level methods, how to learn phrase rep- resentations for cross-lingual phrase retrieval is still an open problem. Learning cross-lingual phrase representations is challenging in two as- pects. First, a phrase is a conceptual unit con- taining multiple words, so it is necessary to model the interaction between words, which is not con- sidered in word-level methods. Second, …

Content truncated to 500 words