HW-TSC’s Participation in the WMT 2020 News Translation Shared Task

Proceedings of the 5th Conference on Machine Translation (WMT), pages 293–299 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 293 HW-TSC’s Participation in the WMT 2020 News Translation Shared Task Daimeng Wei1, Hengchao Shang1, Zhanglin Wu1, Zhengzhe Yu1, Liangyou Li2, Jiaxin Guo1, Minghan Wang1, Hao Yang1, Lizhi Lei1, Ying Qin1, Shiliang Sun3, 1Huawei Translation Service Center, Beijing, China 2Huawei Noah’s Ark Lab, Hong Kong, China 3East China Normal University, Shanghai, China {weidaimeng,shanghengchao,wuzhanglin2,yuzhengzhe,liliangyou, guojiaxin1,wangminghan,yanghao30,leilizhi,qinying}@huawei.com slsun@cs.ecnu.edu.cn Abstract This paper presents our work in the WMT 2020 News Translation Shared Task. We par- ticipate in 3 language pairs including Zh/En, Km/En, and Ps/En and in both directions under the constrained condition. We use the standard Transformer-Big model as the baseline and ob- tain the best performance via two variants with larger parameter sizes. We perform detailed pre-processing and filtering on the provided large-scale bilingual and monolingual dataset. Several commonly used strategies are used to train our models such as Back Translation, En- semble Knowledge Distillation, etc. We also conduct experiment with similar language aug- mentation, which lead to positive results, al- though not used in our submission. Our sub- mission obtains competitive results in the final evaluation. 1 Introduction This paper introduces our work for the WMT 2020 News Translation Shared Task. We participate in three language pairs including Chinese/English (Zh/En), Khmer/English (Km/En), Pashto/English (Ps/En) and in both directions. After observation, we consider that the officially provided dataset has the acceptable size and quality therefore only par- ticipate in the constrained evaluation. Our method is mainly based on previous works but with fine- grained data cleaning techniques and language pair specific optimizations. For each language pair, we perform careful multi- step cleaning on the provided dataset and only keep a high-quality subset for training. At the same time, several strategies are tested in a pipeline including Back-Translation (Edunov et al., 2018), Ensemble Knowledge Distillation (Freitag et al., 2017; Li et al., 2019), Forward Translation (Wu et al., 2019), Fine-Tuning (Sun et al., 2019), and Ensemble and Re-ranking (Ng et al., 2019a). Due to the page limitation, we mainly introduce our methods and experiments on the Zh-En and En- Zh language pairs. Most of these methods are also employed on the Km/En and Ps/En pairs. Special optimizations regarding different language will be introduced separately. 2 Data In this section, we describe the size and source of the dataset as well as our cleaning and filtering techniques. 2.1 Data Source 2.1.1 Zh/En We use both bilingual and monolingual text to train the model. Regarding bilingual text, we merge the data from CCMT (7M), Wiki Titles v1 (1M), News Commentary v15 (0.4M) and a subset of UN Parallel Corpus (9M). We also select 10 million of Zh and En monolingual text from Xin Hua, XMU and News crawl respectively for back translation. 2.1.2 Km/En We use the Para Crawl v5.1 (4.17M), Khmer and Pashto parallel data (0.29M) as the bitext corpus, and select 10M monolingual text from Common Crawl and news crawl 2018 for Km …

Content truncated to 500 words