Tencent Neural Machine Translation Systems for the WMT20 News Translation Task

Proceedings of the 5th Conference on Machine Translation (WMT), pages 313–319 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 313 Tencent Neural Machine Translation Systems for the WMT20 News Translation Task Shuangzhi Wu∗ SPPD of Tencent Xing Wang∗ Tencent AI Lab Longyue Wang∗ Tencent AI Lab Fangxu Liu∗ SPPD of Tencent Jun Xie SPPD of Tencent Zhaopeng Tu Tencent AI Lab Shuming Shi Tencent AI Lab Mu Li SPPD of Tencent Abstract This paper describes Tencent Neural Machine Translation systems for the WMT 2020 news translation tasks. We participate in the shared news translation task on English ↔ Chinese and English → German language pairs. Our systems are built on deep Transformer and sev- eral data augmentation methods. We propose a boosted in-domain finetuning method to im- prove single models. Ensemble is used to com- bine single models and we propose an itera- tive transductive ensemble method which can further improve the translation performance based on the ensemble results. We achieve a BLEU score of 36.8 and the highest chrF score of 0.648 on Chinese → English task. 1 Introduction Recently, Transformer (Vaswani et al., 2017), that depends on self-attention mechanism , has signifi- cantly improved the translation quality. It is widely used as basic Neural Machine Translation (NMT) models in previous WMT translation tasks (Wang et al., 2018b; Li et al., 2019; Sun et al., 2019). In this year’s translation task, our Tencent Transla- tion team participated in three WMT2020 shared news translation tasks, including Chinese → En- glish, English → Chinese and English → German. For the three tasks, we use similar model architec- tures and training strategies. Four structures are used and all of them are based on deep transformer which are proven more effective than the standard Transformer-big models (Li et al., 2019). In terms of data augmentation, we adopt R2L training (Zhang et al., 2019) to all the tasks. Mono- lingual data is only used in English → German task following the back-translation manner (Sennrich et al., 2016b). Different from the standard back- translation, we add noise to the synthetic source ∗ Equal contribution. Correspondence to {frostwu, brightxwang, vinnylywang, fangxuliu}@tencent.com. sentence in order to take advantage of large-scale monolingual text. In addition, we add a special token to the synthetic source sentence to help the model better distinguish the bilingual data and syn- thetic data. The in-domain finetuning (Sun et al., 2019) is very effective in our three experiments and specially, we propose a boosted finetuning method for English ↔ Chinese tasks. We also take advan- tage of the combination methods to further improve the translation quality. The “greedy search ensem- ble algorithm” (Li et al., 2019) is used to select the best combinations from single models. Then for English ↔ Chinese tasks we propose an iterative transductive ensemble (ITE) method based on the translation results of the ensemble models. For En- glish → German task, we apply the noise channel model for re-ranking (Yee et al., 2019). This paper was structured as follows: …

Content truncated to 500 words