HW-TSC’s Participation in the WMT 2021 Triangular MT Shared Task

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 325–330 November 10–11, 2021. ©2021 Association for Computational Linguistics 325 HW-TSC’s Participation in the WMT 2021 Triangular MT Shared Task Zongyao Li, Daimeng Wei, Hengchao Shang, Xiaoyu Chen, Zhanglin Wu, Zhengzhe Yu, Jiaxin Guo, Minghan Wang, Lizhi Lei, Min Zhang, Hao Yang, Ying Qin, Huawei Translation Service Center, Beijing, China {lizongyao,weidaimeng,shanghengchao,chenxiaoyu35, wuzhanglin2,yuzhengzhe,guojiaxin1,wangminghan, leilizhi,zhangmin186,yanghao30,qinying}@huawei.com Abstract This paper presents the submission of Huawei Translation Service Center (HW-TSC) to WMT 2021 Triangular MT Shared Task. We participate in the Russian-to-Chinese task un- der the constrained condition. We use Trans- former architecture and obtain the best per- formance via a variant with larger parameter sizes. We perform detailed data pre-processing and filtering on the provided large-scale bilin- gual data. Several strategies are used to train our models, such as Multilingual Translation, Back Translation, Forward Translation, Data Denoising, Average Checkpoint, Ensemble, Fine-tuning, etc. Our system obtains 32.5 BLEU on the dev set and 27.7 BLEU on the test set, the highest score among all submis- sions. 1 Introduction This paper introduces our submission to the WMT21 Triangular task. We adopt Transformer (Vaswani et al., 2017) architecture and strictly obey the constrained condition in terms of data usage. On one hand, we perform multiple data filtering strategies to enhance data quality; on the other hand, we leverage multilingual model (Johnson et al., 2017), pivot language, forward (Wu et al., 2019) and back translation (Edunov et al., 2018), and data denoising (Wang et al., 2018) strategies to further enhance training effects. In addition, we also adopt fine-tuning (Sun et al., 2019) and ensem- ble (Garmash and Monz, 2016), two widely used strategies, to further enhance system performance. We compare and contrast different strategies based on our experiment results and give our analysis accordingly. The overall training process is illustrated in Fig- ure 1. Section 2 mainly focuses on our training techniques, including model architecture, data pro- cessing and training strategies. Section 3 describes our experiment settings and training process. Sec- tion 4 presents the experiment results while section 5 analyze how our multilingual, data denoise and data augmentation strategies influence system per- formances. 2 Method 2.1 Model Architecture Our system uses Transformer (Vaswani et al., 2017) model architecture, which adopts full self-attention mechanism to realize algorithm parallelism, accel- erate model training speed, and improve transla- tion quality. In this shared task, Transformer-Deep (Wang et al., 2019) is used, which features 35-layer encoder, 6-layer decoder, 768 dimensions of word vector, 3072-hidden-state, 16-head self-attention, and pre-norm. 2.2 Data Processing an Augmentation We strictly comply with the constrained condition and use only the officially provided data. 2.2.1 Data Filtering We perform the following steps to cleanse all data: • Filter out repeated sentences (Khayrallah and Koehn, 2018; Ott et al., 2018). • Convert XML escape characters. • Normalize punctuations using Moses (Koehn et al., 2007). • Delete html tags, non-UTF-8 characters, uni- code characters and invisible characters. • Filter out sentences with mismatched paren- theses and quotation marks; sentences of …

Content truncated to 500 words