Fast and Accurate Neural Machine Translation with Translation Memory

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 3170–3180 August 1–6, 2021. ©2021 Association for Computational Linguistics 3170 Fast and Accurate Neural Machine Translation with Translation Memory Qiuxiang He1 Guoping Huang2 Qu Cui3 Li Li1∗ Lemao Liu2 1 Southwest University 2 Tencent AI Lab 3 Nanjing University hqxiang@email.swu.edu.cn cuiq@smail.nju.edu.cn {donkeyhuang,redmondliu}@tencent.com lily@swu.edu.cn Abstract It is generally believed that a translation memory (TM) should be beneficial for ma- chine translation tasks. Unfortunately, existing wisdom demonstrates the superiority of TM- based neural machine translation (NMT) only on the TM-specialized translation tasks rather than general tasks, with a non-negligible com- putational overhead. In this paper, we pro- pose a fast and accurate approach to TM-based NMT within the Transformer framework: the model architecture is simple and employs a sin- gle bilingual sentence as its TM, leading to ef- ficient training and inference; and its parame- ters are effectively optimized through a novel training criterion. Extensive experiments on six TM-specialized tasks show that the pro- posed approach substantially surpasses several strong baselines that use multiple TMs, in terms of BLEU and running time. In partic- ular, the proposed approach also advances the strong baselines on two general tasks (WMT news Zh→En and En→De). 1 Introduction A translation memory (TM) is originally collected from the translation history of professional trans- lators, and provides the most similar source-target sentence pairs for the source sentence to be trans- lated (Garcia, 2009; Koehn and Senellart, 2010b; Utiyama et al., 2011; Robinson, 2012; Huang et al., 2021). A TM generally provides valuable transla- tion information particularly for those input sen- tences preferably matching the source sentences in the TM, and many efforts have been devoted to integrating a TM into statistical machine transla- tion (Simard and Isabelle, 2009; Koehn and Senel- lart, 2010a; Ma et al., 2011; Wang et al., 2013; Liu et al., 2019). Recently there are increasing interests in im- proving neural machine translation (NMT) with a ∗Corresponding author. TM (Li et al., 2016; Farajian et al., 2017; Gu et al., 2018; Xia et al., 2019; Bulte and Tezcan, 2019; Xu et al., 2020). Many notable approaches have been proposed to augment an NMT model by using a TM. For example, Zhang et al. (2018) and He et al. (2019) extract scored n-grams from a TM and then reward each partial translation once it matches an extracted n-gram during beam search. Gu et al. (2018) and Xia et al. (2019) use an auxiliary net- work to encode a TM and then integrate it into the NMT architecture. Bulte and Tezcan (2019) and Xu et al. (2020) employ data augmentation to train an NMT model whose training instances are bilingual sentences augmented by their TMs. Despite their improvements on the TM-specialized translation tasks (aka JRC-Acquis corpora) where a TM is very similar to test sentences, they consume consider- able computational overheads in either training or testing, and particularly it is unclear whether they can …

Content truncated to 500 words