Study on Unsupervised Statistical Machine Translation for Backtranslation

Proceedings of Recent Advances in Natural Language Processing, pages 578–582, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_068 578 Study on Unsupervised Statistical Machine Translation for Backtranslation Anush Kumar1, Nihal V. Nayak2, Aditya Chandra1 and Mydhili K. Nair1 1Department of Information Science and Engineering, Ramaiah Institute of Technology 2Department of Computer Science, Brown University anush070@gmail.com, nihalnayak@brown.edu, adihyachndrt@gmail.com, mydhili.nair@msrit.edu Abstract Machine Translation systems have drastically improved over the years for several language pairs. Monolingual data is often used to gener- ate synthetic sentences to augment the training data which has shown to improve the perfor- mance of machine translation models. In our paper, we make use of an Unsupervised Sta- tistical Machine Translation (USMT) to gen- erate synthetic sentences. Our study com- pares the performance improvements in Neu- ral Machine Translation model when using synthetic sentences from supervised and un- supervised Machine Translation models. Our approach of using USMT for backtranslation shows promise in low resource conditions and achieves an improvement of 3.2 BLEU score over the Neural Machine Translation model. 1 Introduction Neural Machine Translation systems with encoder-decoder architecture have significantly improved the state-of-the-art for several language pairs (Bahdanau et al., 2015; Vaswani et al., 2017). A majority of the systems in WMT18 used a Transformer approach with varying number of encoder-decoder layers (Bojar et al., 2018). Supervised Neural Machine Translation sys- tems are data-hungry as they require huge amounts of aligned parallel corpora (Koehn and Knowles, 2017). Additionally, obtaining parallel corpora is expensive and requires expert knowledge in both - source and target languages. To overcome this bottleneck, recent research has focused on unsu- pervised machine translation where only mono- lingual corpus is required, eliminating the need for a bilingual parallel corpora. The approaches in unsupervised machine translation have shown promise (Lample et al., 2018; Artetxe et al., 2018, 2019). Nonetheless, the performance of traditional Neural Machine Translation is still better than Un- supervised Machine Translation. Often monolingual data is used to further im- prove the performance of a Neural Machine Trans- lation model (Sennrich et al., 2016a; Currey et al., 2017). Backtranslation is one such popu- lar method in which monolingual data is utilized to improve the model performance. In this technique, a model is initially trained in one of the direc- tions (say target to source) and the trained model is used to translate a monolingual corpora to obtain synthetic sentences in the source language. These synthetic sentences are then included in the train- ing set and a new model is trained from source to target. We take advantage of Unsupervised Ma- chine Translation model for backtranslation (i.e. to generate synthetic sentences). In our paper, we use Unsupervised Machine Translation Model ( USMT) to generate the syn- thetic sentences for Russian-English language pair. Our aim is to improve the performance of the machine translation model using synthetic sen- tences from USMT model. Additionally, we look provide information on the settings and scenarios which benefit from USMT model and NMT model based backtranslation. To the best of our knowl- edge, …

Content truncated to 500 words