HW-TSC Translation Systems for the WMT22 Biomedical Translation Task

Proceedings of the Seventh Conference on Machine Translation (WMT), pages 936 - 942 December 7-8, 2022 ©2022 Association for Computational Linguistics HW-TSC Translation Systems for the WMT22 Biomedical Translation Task Zhanglin Wu, Jinlong Yang, Zhiqiang Rao, Zhengzhe Yu, Daimeng Wei, Xiaoyu Chen, Zongyao Li, Hengchao Shang, Shaojun Li, Ming Zhu, Yuanchang Luo, Yuhao Xie, Miaomiao Ma, Ting Zhu, Lizhi Lei, Song Peng, Hao Yang, Ying Qin Huawei Translation Service Center, Beijing, China {wuzhanglin2,yangjinlong7,raozhiqiang,yuzhengzhe,weidaimeng,chenxiaoyu35, lizongyao,shanghengchao,lishaojun18,zhuming47,luoyuanchang,xieyuhao2, mamiaomiao,zhuting20,leilizhi,pengsong2,yanghao30,qinying}@huawei.com Abstract This paper describes the translation sys- tems trained by Huawei translation ser- vices center (HW-TSC) for the WMT22 biomedical translation task in five lan- guage pairs: English↔German (en↔de), English↔French (en↔fr), English↔Chinese (en↔zh), English↔Russian (en↔ru) and Spanish→English (es→en). Our primary sys- tems are built on deep Transformer with a large filter size. We also utilize R-Drop, data di- versification, forward translation, back transla- tion, data selection, finetuning and ensemble to improve the system performance. According to the official evaluation results in OCELoT1 or CodaLab2, our unconstrained systems in en→de, de→en, en→fr, fr→en, en→zh and es→en (clinical terminology sub-track) get the highest BLEU scores among all submissions for the WMT22 biomedical translation task. 1 Introduction Machine translation (MT) refers to the automatic translation of text from one language to another, and the biomedical translation task aims to evaluate the performance of MT systems in the biomedical domain. In this year’s biomedical translation task, our team (HW-TSC) participates in five language pairs, including en↔de, en↔fr, en↔zh, en↔ru and es→en (clinical terminology sub-track). Since the size of in-domain (ID) data is limited, we first use a large amount of out-of-domain (OOD) data to train our baseline neural machine transla- tion (NMT) (Sutskever et al., 2014; Bahdanau et al., 2015; Gehring et al., 2017; Vaswani et al., 2017) system, which is a deep transformer model (Dou et al., 2018; Li et al., 2019) leveraging R-Drop (Wu et al., 2021) training strategy. We then use the collected ID data (except the data from medical 1https://ocelot-wmt22.mteval.org 2https://codalab.lisn.upsaclay.fr/ competitions/6696#results database) to further train the NMT model for do- main transfer. To better use the limited ID training data, we employ data selection to extract ID data from OOD data, in addition to basic data augmen- tation strategies including data diversity, forward translation and back translation. Finally, we use finetuning (Dakwale and Monz, 2017) and model ensemble (Wang et al., 2020b) to further improve model performance in the biomedical domain. This paper is structured as follows: we describe data size and data pre-processing methods in sec- tion 2; the model structure and training methods in section 3; final results in section 4; and conclusion in section 5. 2 Dataset 2.1 Data volume The data size for each language pair for the WMT22 biomedical translation task is shown in Table 1. The OOD bilingual data, used to train our baseline model, comes from the WMT general MT task and our internal corpus; while the ID bilingual and monolingual data, used for transferring the do- main (Yang et al., 2021), come from Biomedical Translation, UFAL …

Content truncated to 500 words