HW-TSC Systems for WMT22 Very Low Resource Supervised MT Task

Proceedings of the Seventh Conference on Machine Translation (WMT), pages 1098 - 1103 December 7-8, 2022 ©2022 Association for Computational Linguistics HW-TSC Systems for WMT22 Very Low Resource Supervised MT Task Shaojun Li, Yuanchang Luo, Daimeng Wei, Zongyao Li, Hengchao Shang, Xiaoyu Chen, Zhanglin Wu, Jinlong Yang, Zhiqiang Rao, Zhengzhe Yu, Yuhao Xie, Lizhi Lei, Hao Yang, Ying Qin Huawei Translation Service Center, Beijing, China {lishaojun18,luoyuanchang,weidaimeng,lizongyao,shanghengchao,chenxiaoyu35, wuzhanglin2,yangjinlong7,raozhiqiang,yuzhengzhe,xieyuhao2, leilizhi,yanghao30,qinying}@huawei.com Abstract This paper describes the submissions of Huawei translation services center (HW-TSC) to the WMT22 Very Low Resource Super- vised MT task. We participate in all 6 su- pervised tracks including all combinations be- tween Upper/Lower Sorbian (Hsb/Dsb) and German (De). Our systems are build on deep Transformer with a large filter size. We use multilingual transfer with German-Czech (De- Cs) and German-Polish (De-Pl) parallel data. We also utilize regularized dropout (R-Drop), back translation, fine-tuning and ensemble to improve the system performance. According to the official evaluation results on OCELoT1, our supervised systems for all 6 language di- rections got the highest BLEU scores among all submissions. Our pre-trained multilingual model for unsupervised De2Dsb and Dsb2De translation also gains the highest BLEU. 1 Introduction In this paper, we describe our very low resource su- pervised MT systems for all combinations between Hsb, Dsb and De. We first select a base pre-trained multilingual model and then fine-tune it. As we fo- cus primarily on the supervised task, we only apply our pre-trained multilingual system with zero-shot for unsupervised task submissions. As show in WMT21 shared task (Libovický and Fraser, 2021), most participants use De-Cs for transfer or combine De-Cs with the low re- source pairs to build a multilingual system. Fine- tuning based on a multilingual pre-trained (Fan et al., 2020) model has shown very promising re- sults for low resource tasks. We add De-Pl data and train our multilingual pre-trained model to transfer the low resource pairs. This paper is structured as follows: we describe our data source and data pre-processing method in section 2. We detail the model structure and 1https://ocelot-wmt22.mteval.org method we used in Section 3. We then present the final experiments in Section 4 and Section 5, and finally we conclude our work in Section 6. 2 Dataset 2.1 Data Source For our base pre-trained multilingual systems, we use all the bilingual data (De-Cs and De-Pl) from the latest version of OPUS. We also sample 20M German monolingual data from news (general) MT task for augmentation. For fine-tuning the systems transfer to our task, we use all the bilingual and monolingual data officially provided without any filtering strategy. We use dev set and test set to- gether for model parameter adjustment and system selection (do not include the blind test data from the previous years). 2.2 Data Pre-processing For all the data mentioned above, we remove dupli- cate sentences (Khayrallah and Koehn, 2018; Ott et al., 2018). For De-Cs and De-Pl, the data pre-processing procedure is as follows: • Remove sentences with mismatched parenthe- ses and quotation marks. • …

Content truncated to 500 words