FJWU participation for the WMT20 Biomedical Translation Task

Proceedings of the 5th Conference on Machine Translation (WMT), pages 849–856 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 849 FJWU participation for the WMT20 Biomedical Translation Task Sumbal Naz1, Sadaf Abdul Rauf1,2, Noor e Hira1, Syeda Abida1 and Sami Ul Haq3 1 Fatima Jinnah Women University, Pakistan 2 Univ. Paris-Saclay, CNRS, LIMSI France 3 National University of Sciences and Technology, Pakistan {sadaf.abdulrauf,sumbalnaz01,noorehira94,sami.haq99}@gmail.com Abstract This paper reports system descriptions for FJWU-NRPU team for participation in the WMT20 Biomedical shared translation task. We focused our submission on exploring the effects of adding in-domain corpora extracted from various out-of-domain sources. Systems were built for French to English using in- domain corpora through fine tuning and selec- tive data training. We further explored BERT based models specifically with focus on effect of domain adaptive subword units. 1 Introduction In this paper, we present Neural Machine Transla- tion (NMT) systems developed by Fatima Jinnah Women University for participation in WMT20, Biomedical shared Translation task. The sys- tems are developed for translating English/French (EN/FR) in both directions for biomedical domain using fairseq (Ott et al., 2019) and BERT (Devlin et al., 2018). To tackle in-domain corpus shortage challenge, selective data training and fine tuning are explored. We focused our submission on in- vestigating the effects of adding in-domain corpora extracted from out-of-domain sources of various domains, objective was to study the effect of do- main non-relatedness in schemes involving data selection through information retrieval or any sen- tence selection method. We further explored BERT based models specifically with focus on effect of domain adaptive subword units. Neural Machine Translation systems have shown substantial growth with the ongoing introduction of new tool kits and training techniques to support de- velopers in training models (Bahdanau et al., 2014; Wu et al., 2016). But the availability and cleaning of domain related corpora to achieve terminology advantage and fluency is still a challenge for many researchers as the accessible corpora is relatively small in size and comparatively noisy. In order to improve in-domain NMT systems, out-of-domain data is used and the most common method is to fine tune pre trained NMT models on in-domain data and selective data training (Hira et al., 2019). Our last years submission presented promising results using selective data training incorporating data retrieved from News Commentary corpus by building two layered RNN systems. We extend our framework to study the quality of retrieved sen- tences from 3 more parallel corpora. We did not restrict to parallel data for mining biomedical sen- tences, rather this year we included monolingual data in our framework and studied the effect of us- ing Back Translations (BT) in our framework. For building NMT models we explored subword units and report the results on using pre-trained BERT fused embedding. 2 Data Selection Architecture Improving translation quality is a challenging task especially for domains where enough in-domain parallel corpus is not available to train a good trans- lation system. To overcome data scarcity prob- lem, several data selection techniques have been …

Content truncated to 500 words