Neural Machine Translation with Contrastive Translation Memories

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3591 - 3601 December 7-11, 2022 ©2022 Association for Computational Linguistics Neural Machine Translation with Contrastive Translation Memories Xin Cheng 1,2, Shen Gao 3, Lemao Liu 4, Dongyan Zhao 1,2,5,6∗, Rui Yan 7∗ 1 Wangxuan Institute of Computer Technology, Peking University 2 Center for Data Science, Peking University 4 Tencent AI Lab 3 School of Computer Science and Technology, Shandong University 5 State Key Laboratory of Media Convergence Production Technology and Systems 6Beijing Institute of General Artificial Intelligence (BIGAI) 7 Gaoling School of Artificial Intelligence, Renmin University of China chengxin1998@stu.pku.edu.cn shengao@sdu.edu.cn zhaody@pku.edu.cn redmondliu@tencent.com ruiyan@ruc.edu.cn Abstract Retrieval-augmented Neural Machine Trans- lation models have been successful in many translation scenarios. Different from previ- ous works that make use of mutually similar but redundant translation memories (TMs), we propose a new retrieval-augmented NMT to model contrastively retrieved translation mem- ories that are holistically similar to the source sentence while individually contrastive to each other providing maximal information gains in three phases. First, in TM retrieval phase, we adopt a contrastive retrieval algorithm to avoid redundancy and uninformativeness of similar translation pieces. Second, in memory encod- ing stage, given a set of TMs we propose a novel Hierarchical Group Attention module to gather both local context of each TM and global context of the whole TM set. Finally, in train- ing phase, a Multi-TM contrastive learning ob- jective is introduced to learn salient feature of each TM with respect to target sentence. Ex- perimental results show that our framework ob- tains improvements over strong baselines on the benchmark datasets. 1 Introduction Translation memory (TM) is basically a database of segmented and paired source and target texts that translators can access in order to re-use previ- ous translations while translating new texts (Chris- tensen and Schjoldager, 2010). For human trans- lators, such similar translation pieces can lead to higher productivity and consistency (Yamada, 2011). For machine translation, early works mainly contributes to employ TM for statistical machine translation (SMT) systems (Simard and Isabelle, 2009; Utiyama et al., 2011; Liu et al., 2012, 2019). Recently, as neural machine translation (NMT) model (Sutskever et al., 2014; Vaswani et al., 2017) has achieved impressive performance in many ∗Corresponding author. TM1: What is your favorite snack ? TM2: What is your favorite car ? TM3: What is your favorite movie ? (a) Greedy Retrieval Source: What is your favorite sport? Similarity 0.97 0.97 0.97 TM1: What is your favorite snack ? TM2: What sport might be your favorite ? TM3: Which sport do you like best ? (b) Contrastive Retrieval 0.97 0.89 0.81 1 1 2 3 2 3 Source Contrastive TM Greedy TM Figure 1: An example of Greedy Retrieval and Con- trastive Retrieval. The similarity score is computed by edit distance detailed in Section 3.1. And the target side of TM is omitted for brevity. translation tasks, there is also an emerging inter- est (Gu et al., 2018) in retrieval-augmented NMT model. The key …

Content truncated to 500 words