Optum’s Submission to WMT22 Biomedical Translation Tasks

Proceedings of the Seventh Conference on Machine Translation (WMT), pages 925 - 929 December 7-8, 2022 ©2022 Association for Computational Linguistics Optum’s Submission to WMT22 Biomedical translation tasks Sahil Manchanda sahil_manchanda@optum.com Saurabh Bhagwat saurabh_bhagwat@optum.com Abstract This paper describes Optum’s submission to the Biomedical Translation task of the seventh conference on Machine Translation (WMT22). The task aims at promoting the development and evaluation of machine translation systems in their ability to handle challenging domain- specific biomedical data. We made submissions to two sub-tracks of ClinSpEn 2022, namely, ClinSpEn-CC (clinical cases) and ClinSpEn- OC (ontology concepts). These sub-tasks aim to test translation from English to Spanish. Our approach involves fine-tuning a pre-trained transformer model using in-house clinical do- main data and the biomedical data provided by WMT. The fine-tuned model results in a test BLEU score of 38.12 in the ClinSpEn-CC (clinical cases) subtask, which is a gain of 1.23 BLEU compared to the pre-trained model. 1 Introduction The quality of Neural Machine Translation (NMT) was boosted by the use of Recurrent Neural Net- works (RNN) for machine translation. In this ap- proach, the source sentence is fed to an encoder which outputs a context vector. This context vector is fed to the decoder to output the target language text (Cho et al., 2014). Some approaches also use Long Short Term Memory (Hochreiter and Schmid- huber, 1997) for this task (Sutskever et al., 2014). Machine Translation (MT) systems after seeing great progress in recent years have been found to be sensitive to synthetic and natural noise in in- put, distributional shift, and adversarial examples (Koehn and Knowles, 2017; Belinkov and Bisk, 2017; Durrani et al., 2019; Anastasopoulos et al., 2019; Michel et al., 2019). Fine-tuning has proven to be a successful technique to carry out this task. One of the most prominent variations is described in (Chu and Wang, 2018), which trains an NMT model on out-of-domain corpora until model con- vergence and then resumes training from step 1 on a mix of in-domain and out-of-domain data. A fine-grained human evaluation research of the transformer based systems and state-of-the-art re- current systems was carried out on the translation from English to Chinese. The evalution results shows reduction in errors by 31 percent and signif- icantly less errors in 10 out of 22 error categories when using Transformer based MT systems. (Ye and Toral, 2020). Another research has shown that improved efficiency and accuracy can be obtained by converting a pre-trained transformer into its ef- ficient recurrent counterpart. A swap procedure is implemented which replaces softmax attention of a pertained transformer with its linear-complexity recurrent alternative followed by fine-tuning. Fine- tuning has proven to help reduce the training cost and improve efficiency and accuracy (Kasai et al., 2021). We took part in WMT 2022 Biomedical trans- lation task from English to Spanish using the fine- tuning approach on the Transformer based models and we describe our efforts in this paper. The paper is structured as follows. The data sets and their preparation is …

Content truncated to 500 words