Huawei’s Submissions to the WMT20 Biomedical Translation Task

Proceedings of the 5th Conference on Machine Translation (WMT), pages 857–861 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 857 Huawei’s Submissions to the WMT20 Biomedical Translation Task Wei Peng1, Jianfeng Liu1, Minghan Wang2, Liangyou Li3, Xupeng Meng1, Hao Yang2, Qun Liu3 1Artificial Intelligence Application Research Center, Huawei Technologies {peng.wei1,liujianfeng,mengxupeng}@huawei.com 2Huawei Translation Service Center, Huawei Technologies {wangminghan,yanghao30}@huawei.com 3Noah’s Ark Lab, Huawei Technologies {liliangyou,qun.liu}@huawei.com Abstract This paper describes Huawei’s submissions to the WMT20 biomedical translation shared task. Apart from experimenting with fine- tuning on domain-specific bitexts, we explore effects of in-domain dictionaries on enhanc- ing cross-domain neural machine translation performance. We utilize a transfer learning strategy through pre-trained machine trans- lation models and extensive scope of engi- neering endeavors. Four of our ten sub- missions achieve state-of-the-art performance according to the official automatic evalu- ation results, namely translation directions on English⇔French, English→German and English→Italian. 1 Introduction Neural machine translation (NMT) models built upon the Transformer architecture (Vaswani et al., 2017) start to dominate the leader board of WMT biomedical shared tasks in recent years (Bawden et al., 2019). In-domain data (parallel and monolin- gual corpora) have been widely used in finetuning general domain NMT models. Despite ongoing improvements on the translation quality observed from recent biomedical shared tasks, domain adap- tation remains an open problem. The in-domain data is hard to obtain and, as a consequence, greatly limits the cross-domain translation capability an NMT model can offer. Domain terminologies, on the other hand, are regarded as critical resources to improve the quality of machine translation by miti- gating effects of scarce in-domain bitexts (Bawden et al., 2019). However, few research works lever- age domain-specific terminologies (or dictionaries) in training cross-domain NMT systems. In this paper, we present the system architecture and research approaches underpinning Huawei’s submissions to the WMT20 biomedical translation task. We implement two NMT systems to max- imize the performances of the shared task. The system I is an in-house NMT system built upon the transformer-big architecture (Vaswani et al., 2017) and trained using general domain data. We explore means to enhance cross-domain coverage of an NMT model by finetuning the NMT model with in-domain bitexts. We also investigate the effects of domain dictionaries in this domain adap- tation process. Reusing pre-trained models has been regarded as an efficient way of transfer learn- ing. Pre-trained NMT models (Ng et al., 2019) are adopted in the system II to this end. All NMT systems are evaluated against the test set released in the WMT19 biomedical shared task. We submitted translated results for a total of ten language directions between English (EN) and other five languages including French (FR), German (DE), Italian (IT), Russian (RU) and Chi- nese (ZH). Four of the submissions achieve the best BLEU scores according to the official auto- matic evaluation results. Substantial increases in BLEU scores are recorded in translation directions of DE→EN (+3.9 BLEU), ZH→EN (+3.5 BLEU), and EN→DE (+2.8 BLEU) compared to our sub- missions last year (Peng et al., 2019). The im- …

Content truncated to 500 words