HW-TSC’s Participation in the WMT 2021 Large-Scale Multilingual Translation Task

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 456–463 November 10–11, 2021. ©2021 Association for Computational Linguistics 456 HW-TSC’s Participation in the WMT 2021 Large-Scale Multilingual Translation Task Zhengzhe Yu, Daimeng Wei, Zongyao Li, Hengchao Shang, Zhanglin Wu, Xiaoyu Chen, Jiaxin Guo, Minghan Wang, Lizhi Lei, Min Zhang, Hao Yang, Ying Qin, Huawei Translation Service Center, Beijing, China {yuzhengzhe,weidaimeng,lizongyao,shanghengchao, chenxiaoyu35,wuzhanglin2,guojiaxin1,wangminghan, leilizhi,zhangmin186,yanghao30,qinying}@huawei.com Abstract This paper presents the submission of Huawei Translation Services Center (HW-TSC) to the WMT 2021 Large-Scale Multilingual Transla- tion Task. We participate in Small Track #2, including 6 languages: Javanese (Jv), Indone- sian (Id), Malay (Ms), Tagalog (Tl), Tamil (Ta) and English (En) with 30 directions under the constrained condition. We use Transformer ar- chitecture and obtain the best performance via multiple variants with larger parameter sizes. We train a single multilingual model to trans- late all the 30 directions. We perform detailed pre-processing and filtering on the provided large-scale bilingual and monolingual datasets. Several commonly used strategies are used to train our models, such as Back Transla- tion, Forward Translation, Ensemble Knowl- edge Distillation, Adapter Fine-tuning. Our model obtains competitive results in the end. 1 Introduction This paper introduces our submission to the WMT 2021 Large-Scale Multilingual Translation Task. We participate in Small Track #2, including 6 lan- guages: Javanese (Jv), Indonesian (Id), Malay (Ms), Tagalog (Tl), Tamil (Ta) and English (En) with 30 directions. We consider that the officially pro- vided dataset has the acceptable size and quality and therefore only participate in the constrained evaluation. Our method is mainly based on pre- vious works but with fine-grained data cleaning techniques and a multi-step multilingual training strategy. For each language pair, we perform multi-step data cleaning on the provided dataset and only keep a high-quality subset for training. At the same time, several training strategies are tested in a pipeline, including Backward (Edunov et al., 2018) and For- ward (Wu et al., 2019a) Translation, Multilingual Translation (Johnson et al., 2017), Iterative Joint Training (Zhang et al., 2018), Ensemble Knowl- edge Distillation (Freitag et al., 2017; Li et al., 2019), Adapter Fine-Tuning (Bapna et al., 2019), and Ensemble (Garmash and Monz, 2016). Based on the task requirements, we train a single multilingual model that translates all 30 directions. We refer to (Johnson et al., 2017) and employ lan- guage tags (Wu et al., 2021). By combining mul- tiple strategies, our model achieves considerable quality improvements in all directions. Section 2 focuses on our data processing strate- gies while section 3 describes our training tech- niques, including model architecture and the iter- ative training strategy, etc. Section 4 explains our experiment settings and training processes and sec- tion 5 presents our experiment results. 2 Data 2.1 Data Source For all language pairs, we follow the constrained data requirements and take full advantage of the bilingual and monolingual training data available. Table 1 lists the data sizes of each language pair before and after filtering. 2.2 Data Pre-processing We conduct the following steps to pre-process the data: …

Content truncated to 500 words