Tencent AI Lab Machine Translation Systems for WMT20 Chat Translation Task

Proceedings of the 5th Conference on Machine Translation (WMT), pages 483–491 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 483 Tencent AI Lab Machine Translation Systems for WMT20 Chat Translation Task Longyue Wang§ Zhaopeng Tu§ Xing Wang§ Li Ding†∗ Liang Ding‡∗ Shuming Shi§ §Tencent AI Lab {vinnylywang,zptu,brightxwang,shumingshi}@tencent.com †Hong Kong Polytechnic University dingli@oppo.com ‡The University of Sydney ldin3097@uni.sydney.edu.au Abstract This paper describes the Tencent AI Lab’s submission of the WMT 2020 shared task on chat translation in English⇔German. Our neu- ral machine translation (NMT) systems are built on sentence-level, document-level, non- autoregressive (NAT) and pretrained models. We integrate a number of advanced techniques into our systems, including data selection, back/forward translation, larger batch learn- ing, model ensemble, finetuning as well as sys- tem combination. Specifically, we proposed a hybrid data selection method to select high- quality and in-domain sentences from out-of- domain data. To better capture the source con- texts, we exploit to augment NAT models with evolved cross-attention. Furthermore, we ex- plore to transfer general knowledge from four different pre-training language models to the downstream translation task. In general, we present extensive experimental results for this new translation task. Among all the partici- pants, our German⇒English primary system is ranked the second in terms of BLEU scores. 1 Introduction Although neural machine translation (NMT, Bah- danau et al., 2015; Vaswani et al., 2017; Gehring et al., 2017) has achieved great progress in recent years, translating conversational text is still a chal- lenging task due to its inherent characteristics such as discourse awareness (Maruf et al., 2018; Wang et al., 2019), informality (Wang et al., 2018; Yang et al., 2019) and personality (Mirkin et al., 2015; Wang et al., 2016). This is a task-oriented chat translation task (Wang et al., 2017a; Farajian et al., 2020), which aims to translating conversations be- tween customers and agents. As a customer and an agent can respectively natively speak in German ∗ This work was conducted when Li Ding and Liang Ding were interning at Tencent AI Lab. Li Ding is now working at OPPO Research Institute. and English, the systems should translate the cus- tomer’s utterances in German⇒English (De⇒En) while the agent’s in German⇐English (De⇐En). In this paper, we present our submission to the novel task in De⇔En. We explore a breadth of established techniques for building Chat NMT sys- tems. Specifically, our systems are based on the self-attention networks including both sentence- and document-level Transformer (Vaswani et al., 2017; Wang et al., 2017b). Besides, we investi- gated non-autoregressive translation (NAT) mod- els augmented with our recently proposed evolved cross-attention (Ding et al., 2020). Technically, we used the most recent effective strategies including back/forward translation, data selection, domain adaptation, batch learning, finetuning, model en- semble and system combination. Particularly, we proposed a multi-feature data selection on large general-domain data. We not only use three lan- guage models (i.e. n-gram, Transformer and BERT based LMs) to filter low-quality sentences, but also employ feature decay algorithms (FDA, Bic¸ici and Yuret, 2011) to select domain-relevant data. …

Content truncated to 500 words