OneAligner: Zero-shot Cross-lingual Transfer with One Rich-Resource Language Pair for Low-Resource Sentence Retrieval

Findings of the Association for Computational Linguistics: ACL 2022, pages 2869 - 2882 May 22-27, 2022 c⃝2022 Association for Computational Linguistics OneAligner: Zero-shot Cross-lingual Transfer with One Rich-Resource Language Pair for Low-Resource Sentence Retrieval Tong Niu Kazuma Hashimoto Yingbo Zhou Caiming Xiong Salesforce Research {tniu, k.hashimoto, yingbo.zhou, cxiong}@salesforce.com Abstract Aligning parallel sentences in multilingual cor- pora is essential to curating data for down- stream applications such as Machine Transla- tion. In this work, we present OneAligner, an alignment model specially designed for sen- tence retrieval tasks. This model is able to train on only one language pair and transfers, in a cross-lingual fashion, to low-resource lan- guage pairs with negligible degradation in per- formance. When trained with all language pairs of a large-scale parallel multilingual corpus (OPUS-100), this model achieves the state-of- the-art result on the Tateoba dataset, outper- forming an equally-sized previous model by 8.0 points in accuracy while using less than 0.6% of their parallel data. When finetuned on a single rich-resource language pair, be it English-centered or not, our model is able to match the performance of the ones finetuned on all language pairs under the same data bud- get with less than 2.0 points decrease in accu- racy. Furthermore, with the same setup, scal- ing up the number of rich-resource language pairs monotonically improves the performance, reaching a minimum of 0.4 points discrepancy in accuracy, making it less mandatory to collect any low-resource parallel data. Finally, we con- clude through empirical results and analyses that the performance of the sentence alignment task depends mostly on the monolingual and parallel data size, up to a certain size threshold, rather than on what language pairs are used for training or evaluation. 1 Introduction Cross-lingual sentence retrieval aims at aligning parallel sentence pairs that are translations of each other from unlabeled multilingual documents. Such mined data can be used in multiple down- stream applications such as Machine Translation and cross-lingual Word Sense Disambiguation (Fan et al., 2020; Tran et al., 2020; Schwenk et al., 2021a,b). Even under a half-automated setting with human-in-the-loop, a faithful aligner can help nar- row down the candidate pool so that humans do not need to deal with an enormous search space such as cross-lingual web-document pairs (El-Kishky et al., 2020) or the entire internet. A retrieval model has also been used to filter existing parallel corpora to improve their quality (Schwenk, 2018) or to per- form Quality Estimation (Fomicheva et al., 2020) where the reference translations are not available. For sentence retrieval tasks, a majority of re- cent work is either completely unsupervised (Hu et al., 2020; Tran et al., 2020; Lewis et al., 2020) or leverages all parallel data available (Artetxe and Schwenk, 2019; Ouyang et al., 2021), sometimes to the extent of 879 language pairs (Luo et al., 2021). The unsupervised approach has the benefit of not collecting any parallel data; yet it usually achieves relatively low accuracies on standard benchmark datasets such as Tatoeba (Artetxe and Schwenk, 2019), which evaluates on …

Content truncated to 500 words