Document Level NMT of Low-Resource Languages with Backtranslation

Proceedings of the 5th Conference on Machine Translation (WMT), pages 442–446 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 442 Document Level NMT of Low-Resource Languages with Backtranslation Sami Ul Haq1, Sadaf Abdul Rauf2,3, Arslan Shoukat1 and Abdullah Saeed4 1 National University of Sciences and Technology, Pakistan 2 Fatima Jinnah Women University, Pakistan 3 LIMSI-CNRS, France 4 COMSATS University, Pakistan {sadaf.abdulrauf,abdullahsaeed98}@gmail.com {sami.ulhaq,arslanshaukat}@ceme.nust.edu.pk Abstract This paper describes our system submission to WMT20 shared task on similar language trans- lation. We examined the use of document- level neural machine translation (NMT) sys- tems for low-resource, similar language pair Marathi−Hindi. Our system is an extension of state-of-the-art Transformer architecture with hierarchical attention networks to incorporate contextual information. Since, NMT requires large amount of parallel data which is not avail- able for this task, our approach is focused on utilizing monolingual data with back transla- tion to train our models. Our experiments re- veal that document-level NMT can be a rea- sonable alternative to sentence-level NMT for improving translation quality of low resourced languages even when used with synthetic data. 1 Introduction With the widespread use of MT systems in com- mercial and research community, there is an in- creased attention to train NMT models for direct translation between language pairs other than En- glish Barrault et al. (2019). This is because of the growing need to translate between pairs of simi- lar languages without considering English as pivot language. The task is to overcome the challenge of limited availability of parallel data by exploit- ing the advantages of similarity between languages when building machine translation models. Similar languages have the advantage of having some mag- nitude of common information such as lexical and semantic structures. A number of research studies have been published to exploit commonalities when translating text between close language pairs Pour- damghani and Knight (2017); Lakew et al. (2018); Costa-juss`a (2017). This paper describes our system submission at WMT shared Similar Language Translation task1 1http://www.statmt.org/wmt20/similar.html which focuses on improving translation quality of similar languages in low-resource setting, the detail of task is provided in Barrault et al. (2019). This year’s task includes five pairs of languages from three different language families i.e. Indo-Aryan, Romance and South-Slavic languages; we partic- ipated for Hindi-Marathi language pair. Since we are using NMT which requires large bitext, we need to alleviate this specific problem of bitext short- age. Sennrich et al. (2016) introduced an approach to utilize monolingual data using back translation. This requires a machine translation system in oppo- site direction to generate synthetic parallel corpora from target side monolingual text. Our work is an attempt to investigate the trans- lation of a similar language pair (Marathi-Hindi) using document-level NMT and back translation. We participated under team name ”FJWU NUST”. We submitted one constrained system i.e. we only used the parallel and monolingual data provided by WMT202 organizers to train and evaluate our mod- els. We train and evaluate NMT systems in both directions (i.e. HI⇒MR and MR⇒HI) but our sub- mission …

Content truncated to 500 words