FJWU Participation for the WMT21 Biomedical Translation Task

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 857–862 November 10–11, 2021. ©2021 Association for Computational Linguistics 857 FJWU participation for the WMT21 Biomedical Translation Task Sumbal Naz1, Sadaf Abdul Rauf1,2 and Sami ul Haque3 1 Fatima Jinnah Women University, Pakistan 2 Univ. Paris-Saclay, LISN-CNRS, France 3 National University of Science and Technology, Pakistan {sadaf.abdulrauf,sumbalnaz01}@gmail.com Abstract In this paper we present the FJWU’s system submitted to the biomedical shared task at WMT21. We prepared state-of-the-art mul- tilingual neural machine translation systems for three languages (i.e. German, Spanish and French) with English as target language. Our NMT systems based on Transformer ar- chitecture, were trained on combination of in- domain and out-domain parallel corpora devel- oped using Information Retrieval (IR) and do- main adaptation techniques. 1 Introduction Due to vast availability of multilingual informa- tion, Neural Machine Translation (NMT) systems have achieved remarkable growth over Statistical Machine Translation (SMT) systems. Although the amount of training resources has significantly increased in the past few years but availability of large in-domain parallel data is still a challenging task. Performance of NMT system may quickly degrade as soon as the application domain devi- ates from training domain. Domain adaptation (Koehn and Schroeder, 2007) is a promising active research topic to enhance the translation quality when faced with data scarcity issues. In domain adaptation, initially large amount of parallel out-of- domain corpora is utilized for training NMT mod- els and then fine-tuning is performed on small in- domain data for adapting to novel domains (Freitag and Al-Onaizan, 2016; Luong and Manning, 2015). Fine-tuning does not require building system from scratch, instead it is fast and efficient method of in- tegrating in-domain data. An NMT model already trained on general domain data is further fine-tuned on in-domain data with less time and effort (Chu et al., 2017; Hira et al., 2019). Training MT sys- tems on back-translated data is a proven domain adaptation method (Abdul Rauf et al., 2020; Sen- nrich et al., 2015), where synthetic parallel data is combined with original data to generate large in- domain training corpus. In addition, information re- trieval (IR) technique to extract relevant sentences from out-of-domain corpus has shown promising results to overcome data scarcity (Naz et al., 2020). NMT system incorporating multiple languages into single model is known as multilingual NMT (MNMT) (Dabre et al., 2020). Multilingual NMT systems are gaining popularity due to effective use of available resources and boosting translation qual- ity with Translation Knowledge Transfer (Pan and Yang, 2009). In this paper, we present study on adapting MNMT systems (Multiway many-to-one) for trans- lating English (EN) language from French (FR), German (DE) and Spanish (ES) using fairseq (Ott et al., 2019) implementation of Transformer model. Our main focus is to investigate the effect on EN translation in Biomedical domain using multilin- gual NMT systems. We have also explored the do- main adaptation for fine-tuning of bilingual NMT models into multilingual NMT models using out- domain and in-domain corpora. Furthermore, we show the effectiveness …

Content truncated to 500 words