HW-TSC’s Submissions to the WMT21 Biomedical Translation Task

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 879–884 November 10–11, 2021. ©2021 Association for Computational Linguistics 879 HW-TSC’s Submissions to the WMT21 Biomedical Translation Task Hao Yang1, Zhanglin Wu1, Zhengzhe Yu1, Xiaoyu Chen1, Daimeng Wei1, Zongyao Li1, Hengchao Shang1, Jiaxin Guo1, Minghan Wang1, Lizhi Lei1, Chuanfei Xu1, Min Zhang1, Ying Qin1, 1Huawei Translation Service Center, Beijing, China {yanghao30,wuzhanglin2,yuzhengzhe,chenxiaoyu35,weidaimeng, lizongyao,shanghengchao,guojiaxin1,wangminghan, leilizhi,xuchuanfei,zhangmin186,qinying}@huawei.com Abstract This paper describes the submission of Huawei Translation Service Center (HW-TSC) to WMT21 biomedical translation task in two language pairs: Chinese↔English and German↔English (Our registered team name is HuaweiTSC). Technical details are intro- duced in this paper, including model frame- work, data pre-processing method and model enhancement strategies. In addition, us- ing the wmt20 OK-aligned biomedical test set, we compare and analyze system perfor- mances under different strategies. On WMT21 biomedical translation task, Our systems in English→Chinese and English→German di- rections get the highest BLEU scores among all submissions according to the official evalu- ation results. 1 Introduction We have witnessed great progress made by neu- ral machine translations (Bahdanau et al., 2015; Vaswani et al., 2017) in recent years. However, domain adaptation remains to be a tough issue. As noted by Koehn and Knowles (Koehn and Knowles, 2017), translations by NMT systems in out-of- domain scenarios are relatively poor, and high- quality data in specific domains are difficult to obtain, which pose great challenges to certain trans- lation tasks (e.g. biomedical translation). To ad- dress the domain adaptation issue, on one hand, we leverage data diversification (Nguyen et al., 2020), forward translation (Wu et al., 2019) and back trans- lation (Sennrich et al., 2016a; Edunov et al., 2018) to generate synthetic in-domain corpora. On the other hand, fine-tuning (Sun et al., 2019) and en- semble (Freitag et al., 2017; Li et al., 2019) are used to further enhance system performances in the biomedical domain. We introduce our data strategy in section 2, and model architectures as well as model enhancement techniques in section 3. Section 4 presents experi- mental results of both language pairs on the wmt20 OK-aligned biomedical test set. Section 5 is a con- clusion of our work. 2 Dataset 2.1 Data Source Our baseline model is trained with out-of-domain WMT21 news data. The sizes of bilingual and monolingual data for Chinese↔English and German↔English language pairs are shown in Ta- ble 1. With regard to in-domain data, we use both the bilingual data and monolingual data provided by the WMT21 Biomedical Translation Shared task. For German↔English task, we select Biomed- ical Translation and UFAL Medical Corpus as in-domain training data. Besides, 21.43M in- house English monolingual data are used. For Chinese↔English task, the used in-house data includes: 1.35M parallel data, 21.43M English monolingual data, and 36.11M Chinese monolin- gual data. Table 2 shows the details of data in the biomedical domain for German↔English and Chinese↔English tasks. 2.2 Data Pre-processing Our data pre-processing methods include: • Filter out repeated sentences (Khayrallah and Koehn, 2018; Ott et al., 2018). • Normalize punctuations using Moses (Koehn …

Content truncated to 500 words