HW-TSC’s Submissions to the WMT 2022 General Machine Translation Shared Task

Proceedings of the Seventh Conference on Machine Translation (WMT), pages 403 - 410 December 7-8, 2022 ©2022 Association for Computational Linguistics HW-TSC’s Submissions to the WMT 2022 General Machine Translation Shared Task Daimeng Wei, Zhiqiang Rao, Zhanglin Wu, Shaojun Li, Yuanchang Luo, Yuhao Xie, Xiaoyu Chen, Hengchao Shang, Zongyao Li, Zhengzhe Yu, Jinlong Yang, Miaomiao Ma, Lizhi Lei, Hao Yang, Ying Qin, Huawei Translation Service Center, Beijing, China {weidaimeng,raozhiqiang,wuzhanglin2,lishaojun18,luoyuanchang, xieyuhao2,chenxiaoyu35,shanghengchao,lizongyao,yuzhengzhe, yangjinlong7,mamiaomiao,leilizhi,yanghao30,qinying}@huawei.com Abstract This paper presents the submissions of Huawei Translate Services Center (HW-TSC) to the WMT 2022 General Machine Translation Shared Task. We participate in 6 language pairs, including Zh↔En, Ru↔En, Uk↔En, Hr↔En, Uk↔Cs and Liv↔En. We use Transformer ar- chitecture and obtain the best performance via multiple variants with larger parameter sizes. We perform fine-grained pre-processing and fil- tering on the provided large-scale bilingual and monolingual datasets. For medium and high- resource languages, we mainly use data aug- mentation strategies, including Back Transla- tion, Self Training, Ensemble Knowledge Dis- tillation, Multilingual, etc. For low-resource languages such as Liv, we use pre-trained ma- chine translation models, and then continue training with Regularization Dropout (R-Drop). The previous mentioned data augmentation methods are also used. Our submissions ob- tain competitive results in the final evaluation. 1 Introduction This paper introduces our submissions to the WMT 2022 General Machine Translation Shared Task. We participate in 6 language pairs includ- ing Chinese/English (Zh↔En), Russian/English (Ru↔En), Ukrainian/English (Uk↔En), Croat- ian/English (En→Hr), Ukrainian/Czech(Uk↔Cs), and Livonian/English (Liv↔En). For Zh↔En translation, we use additional in-house in-domain data, so the final submission for this language pair is unconstrained. For Liv↔En translation, al- though we did not use additional data, we used M2M-100 (Fan et al., 2020) as the pretrained model, and the final submission is also uncon- strained. All other languages pair participate in the constrained evaluation. Our method is mainly based on previous works (Wei et al., 2020, 2021; Yang et al., 2021) but with fine-grained data cleans- ing techniques and language-specific optimiza- tions. For each language pair, we perform multi-step data cleansing on the provided dataset and only keep a high-quality subset for training. At the same time, several strategies are tested in a pipeline, in- cluding Backward (Edunov et al., 2018) and For- ward (Wu et al., 2019a) Translation, Multilingual Translation (Johnson et al., 2017), Iterative Joint Training (Zhang et al., 2018), R-Drop, Pretrained NMT model, Ensemble Knowledge Distillation (Freitag et al., 2017; Li et al., 2019), Fine-Tuning (Sun et al., 2019), Ensemble (Garmash and Monz, 2016), and Post-Processing. Our system report includes four parts. Section 2 focuses on our data processing strategies while section 3 describes our training details. Section 4 explains our experiment settings and training pro- cesses and section 5 presents the results. 2 Data 2.1 Data Source We obtain bilingual and monolingual data from data sources such as CCMT, UN, ParaCrawl, Wiki- Matrix, WikiTitles, News Commentary, Leipzig Corpora, News Crawl, and Common Crawl. The amount of data we used is shown in Table 1. It should be noted that in order to …

Content truncated to 500 words