HW-TSC’s Participation in the WMT 2021 News Translation Shared Task

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 225–231 November 10–11, 2021. ©2021 Association for Computational Linguistics 225 HW-TSC’s Participation in the WMT 2021 News Translation Shared Task Daimeng Wei, Zongyao Li, Zhanglin Wu, Zhengzhe Yu, Xiaoyu Chen, Hengchao Shang, Jiaxin Guo, Minghan Wang, Lizhi Lei, Min Zhang, Hao Yang, Ying Qin, Huawei Translation Service Center, Beijing, China {weidaimeng,lizongyao,wuzhanglin2,yuzhengzhe, chenxiaoyu35,shanghengchao,guojiaxin1,wangminghan, leilizhi,zhangmin186,yanghao30,qinying}@huawei.com Abstract This paper presents the submission of Huawei Translate Services Center (HW-TSC) to the WMT 2021 News Translation Shared Task. We participate in 7 language pairs, including Zh/En, De/En, Ja/En, Ha/En, Is/En, Hi/Bn, and Xh/Zu in both directions under the con- strained condition. We use Transformer archi- tecture and obtain the best performance via multiple variants with larger parameter sizes. We perform detailed pre-processing and filter- ing on the provided large-scale bilingual and monolingual datasets. Several commonly used strategies are used to train our models, such as Back Translation, Forward Translation, Multi- lingual Translation, Ensemble Knowledge Dis- tillation, etc. Our submission obtains competi- tive results in the final evaluation. 1 Introduction This paper introduces our submission to the WMT 2021 News Translation Shared Task. We par- ticipate in seven language pairs including Chi- nese/English (Zh/En), German/English (De/En), Japanese/English (Ja/En), Hausa/English (Ha/En), Icelandic/English (Is/En), Hindi/Bengali (Hi/Bn), and Xhosa/Zulu (Xh/Zu) in both directions. We consider that the officially provided dataset has the acceptable size and quality and therefore only par- ticipate in the constrained evaluation. Our method is mainly based on previous works but with fine- grained data cleansing techniques and language- specific optimizations. For each language pair, we perform multi-step data cleansing on the provided dataset and only keep a high-quality subset for training. At the same time, several strategies are tested in a pipeline, in- cluding Backward (Edunov et al., 2018) and For- ward(Wu et al., 2019a) Translation, Multilingual Translation (Johnson et al., 2017), Right-to-Left Models (Liu et al., 2016), Iterative Joint Training (Zhang et al., 2018), Ensemble Knowledge Distil- lation (Freitag et al., 2017; Li et al., 2019) , Fine- Tuning (Sun et al., 2019), Ensemble (Garmash and Monz, 2016), and PostProcess. We combined all the techniques mentioned above and the overall training process is shown in Figure 1. Section 2 focuses on our data process- ing strategies while section 3 describes our train- ing techniques, including model architecture and iterative training, etc. Section 4 explains our exper- iment settings and training processes and section 5 presents our experiment results. 2 Data 2.1 Data Source For all language pairs, we follow the constrained data requirements and take full advantages of the bilingual and monolingual training data available. Table 1 lists the data sizes of each language pair before and after filtering. 2.2 Data Pre-processing We use following operations to pre-process the data: • Filter out repeated sentences (Khayrallah and Koehn, 2018; Ott et al., 2018). • Convert XML escape characters. • Normalize punctuations using Moses (Koehn et al., 2007). • Delete html tags, non-UTF-8 characters, uni- code characters and invisible characters. • Filter out sentences with …

Content truncated to 500 words