Huawei AARC’s Submissions to the WMT21 Biomedical Translation Task: Domain Adaption from a Practical Perspective

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 868–873 November 10–11, 2021. ©2021 Association for Computational Linguistics 868 Huawei AARC’s Submissions to the WMT21 Biomedical Translation Task: Domain Adaption from a Practical Perspective Weixuan Wang1 ∗, Wei Peng1 ∗, Xupeng Meng1, Qun Liu2 1Artificial Intelligence Application Research Center, Huawei Technologies {weixuanwang2,peng.wei1,mengxupeng}@huawei.com 2Noah’s Ark Lab, Huawei Technologies {qun.liu}@huawei.com Abstract This paper describes Huawei Artificial Intel- ligence Application Research Center’s neural machine translation systems and submissions to the WMT21 biomedical translation shared task. Four of the submissions achieve state-of-the-art BLEU scores based on the official-released au- tomatic evaluation results (EN→FR, EN↔IT and ZH→EN). We perform experiments to un- veil the practical insights of the involved do- main adaptation techniques, including finetun- ing order, terminology dictionaries, and ensem- ble decoding. Issues associated with overfitting and under-translation are also discussed. 1 Introduction General-purpose machine translation systems have limited capability in addressing domain-specific tasks (Koehn and Knowles, 2017), for example, the WMT biomedical translation shared task, due to the low availability for high-quality in-domain data. In our WMT20 submission, various domain adap- tion technologies (Bawden et al., 2019, 2020) have been applied including practical approaches fine- tuning on general-purpose models, back-translation (Sennrich et al., 2016) and leveraging in-domain dictionaries (Peng et al., 2020b). Despite achieving state-of-the-art (SOTA) BLEU scores for most of the submissions, few efforts were put in place to disclose the practical insights associated with these techniques. This year, the Artificial Intelligence Applica- tion Research Center (AARC) participate in the WMT21 biomedical translation task for eight lan- guage directions between English and other four languages (French, German, Italian, and Chinese). The baseline model is an in-house general-purpose NMT model built upon the transformer-big archi- tecture (Vaswani et al., 2017). Apart from present- ing an overview of the proposed biomedical Neural *Co-first authors. Machine Translation (NMT) system, we investi- gate the practical insights of the involved domain adaptation techniques, including finetuning order, terminology dictionaries, and ensemble decoding. Issues associated with overfitting to in-domain data and under-translation are also discussed. 2 The Data In this section we detail the bilingual and monolin- gual data used in this shared task (Table 1). 2.1 Bilingual Data In-domain bilingual data In all directions, we use the in-domain data (IND) provided by the shared task organizers to finetune the base model. 1 The IND data consists of WMT-released bitexts from Pubmed, UFAL 2 and Medline. 3 We notice that the official release of IND data suffers from issues of misalignment between source and target sentences, and missing target sentences. The translation of a source sentence may be mis- placed in a different line or even appeared in multi- ple lines on the target side. Moreover, a source sen- tence may have not been translated into in a target sentence. A data processing pipeline is developed to address the issues mentioned above (depicted in 3.4). The test data is the official release of the WMT19 shared task. Augmented Bilingual Data We collect in- domain data from TAUS 4 …

Content truncated to 500 words