Huawei BabelTar NMT at WMT22 Biomedical Translation Task: How We Further Improve Domain-specific NMT

Proceedings of the Seventh Conference on Machine Translation (WMT), pages 930 - 935 December 7-8, 2022 ©2022 Association for Computational Linguistics Huawei BabelTar NMT at WMT22 Biomedical Translation Task: How we further improve domain-specific NMT Weixuan Wang , Xupeng Meng, Suqing Yan, Ye Tian, Wei Peng∗ Artificial Intelligence Application Research Center, Huawei Technologies peng.wei1@huawei.com Abstract This paper describes Huawei Artificial Intel- ligence Application Research Center’s neural machine translation system (“BabelTar”). Our submission to the WMT22 biomedical trans- lation shared task covers language directions between English and the other seven languages (French, German, Italian, Spanish, Portuguese, Russian, and Chinese). During the past four years, our participation in this domain-specific track has witnessed a paradigm shift of method- ology from a purely data-driven focus to em- bracing diversified techniques, including pre- trained multilingual NMT models, homograph disambiguation, ensemble learning, and pre- processing methods. We illustrate practical insights and measured performance improve- ments relating to how we further improve our domain-specific NMT system. 1 Introduction The existing mainstream neural machine transla- tion (NMT) system is predominantly data-driven. Our participation in WMT biomedical tasks traced back from 2019 has witnessed pursuits extending beyond this modality. In our WMT20 and WMT21 submissions, various domain adaption technologies (Bawden et al., 2020; Akhbardeh et al., 2021) have been applied including practical approaches fine- tuning on general-purpose models, back-translation (Sennrich et al., 2016a) and leveraging in-domain dictionaries (Peng et al., 2020; Wang et al., 2021). Despite achieving state-of-the-art (SOTA) BLEU scores for most of our submissions in the last two years, under-translation occurred in the “English ↔ Chinese” due to the models’ incapability to handle long sentences (Wang et al., 2021). It was rectified by ensembling the affected model with the baseline, resulting in a decrease in BLEU scores. In addition, the models trained predominately with the general ∗ Corresponding author domain data still face challenges associated with domain adaptation. In this paper, we present practical insights into how we further improve Huawei Artificial Intelli- gence Application Research Center’s neural ma- chine translation system (“BabelTar”) in domain- specific machine translation. This year, our par- ticipation in the WMT22 biomedical translation task covers language directions between “English (EN)” and the other seven languages “German (DE)”, “Spanish (ES)”, “French (FR)”, “Italian (IT)”, “Portuguese (PT)”, “Russian (RU)” and “Chi- nese (ZH)”. More specifically, we adopt in-house general-purposed bilingual NMT models built upon the transformer-big architecture (Vaswani et al., 2017) and a pre-trained multilingual NMT model (M2M100) (Fan et al., 2021) with an M2M100- 418M configuration as baseline models. Finetuned with the in-domain data provided by the organizer, the back-translated monolingual Medline data in English dating before July 2018, the in-domain dic- tionaries enhanced with terminologies, the models can be improved significantly over the last year’s submissions, for example, +1.18 BLEU on “EN → IT” and +1.24 BLEU on “EN → DE”. Leveraging the knowledge learned in addressing the ambigui- ties caused by homographs, we can further boost +0.65 BLEU in the language direction of “EN → ZH”. By optimizing the sequence …

Content truncated to 500 words