Adobe AMPS’s Submission for Very Low Resource Supervised Translation Task at WMT20

Proceedings of the 5th Conference on Machine Translation (WMT), pages 1144–1149 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 1144 Adobe AMPS’s Submission for Very Low Resource Supervised Translation Task at WMT20 Keshaw Singh AI/ML Platform & Solutions, Adobe Inc. Bengaluru, India kessingh@adobe.com Abstract In this paper, we describe our systems sub- mitted to the very low resource supervised translation task at WMT20. We participate in both translation directions for Upper Sorbian- German language pair. Our primary sub- mission is a subword-level Transformer-based neural machine translation model trained on original training bitext. We also conduct several experiments with backtranslation us- ing limited monolingual data in our post- submission work and include our results for the same. In one such experiment, we ob- serve jumps of up to 2.6 BLEU points over the primary system by pretraining on a synthetic, backtranslated corpus followed by fine-tuning on the original parallel training data. 1 Introduction This paper describes our submissions to the shared task on Very Low Resource Supervised Machine Translation at WMT 2020. The task involved a single language pair: Upper Sorbian-German. We submit supervised neural machine translation (NMT) systems for both translation directions, Up- per Sorbian→German and German→Upper Sor- bian. NMT models (Sutskever et al., 2014; Bahdanau et al., 2015; Cho et al., 2014a) have achieved state- of-the-art performance on benchmark datasets for multiple language pairs. A big advantage of such systems over phrase-based statistical machine trans- lation (PBSMT) (Koehn et al., 2003) models is that they can be trained end-to-end. The bulk of the development, however, has been limited to a hand- ful of high-resource language pairs. The primary reason is that training a well-performing NMT sys- tem requires a large amount of parallel training data, which means a lot of equivalent investment in terms of resources. Koehn and Knowles (2017) show that when compared to PBSMT approaches, NMT models need more training data to achieve the same level of performance.1 One of the most popular ways to increase the amount of parallel training data for supervised training is backtrans- lation (Sennrich et al., 2016a). We utilize this ap- proach to improve upon the performance of our baseline models. All of our systems follow the Transformer archi- tecture (Vaswani et al., 2017). Our primary system is a supervised NMT model trained on the original training bitext. We also report our results on experi- ments with backtranslation, which were completed post the shared task and hence not a part of our primary submissions. We use the backtranslated data in two distinct ways - as a standalone parallel corpus, and to create a combined parallel corpus by mixing in a 1:1 ratio with the provided training data. We also report the performance of fine-tuned models originally trained only on the backtrans- lated data. In the following sections, we begin by briefly describing the Transformer architecture and backtranslation. We then discuss our experimental setup as well as our experiments with backtransla- tion. We conclude with a discussion of our results and possible …

Content truncated to 500 words