Combining PBSMT and NMT Back-translated Data for Efficient NMT

Proceedings of Recent Advances in Natural Language Processing, pages 922–931, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_107 922 Combining SMT and NMT Back-Translated Data for Efficient NMT Alberto Poncelas, Maja Popovi´c, Dimitar Shterionov, Gideon Maillette de Buy Wenniger and Andy Way School of Computing, DCU, ADAPT Centre {firstname.lastname}@adaptcentre.ie Abstract Neural Machine Translation (NMT) mod- els achieve their best performance when large sets of parallel data are used for train- ing. Consequently, techniques for aug- menting the training set have become pop- ular recently. One of these methods is back-translation (Sennrich et al., 2016a), which consists on generating synthetic sentences by translating a set of mono- lingual, target-language sentences using a Machine Translation (MT) model. Generally, NMT models are used for back- translation. In this work, we analyze the performance of models when the training data is extended with synthetic data using different MT approaches. In particular we investigate back-translated data generated not only by NMT but also by Statistical Machine Translation (SMT) models and combinations of both. The results reveal that the models achieve the best perfor- mances when the training set is augmented with back-translated data created by merg- ing different MT approaches. 1 Introduction Machine translation (MT) nowadays is heavily dependent on the quantity and quality of train- ing data. The amount of available good-quality parallel data for the desired domain and/or lan- guage pair is often insufficient to reach the re- quired translation performance. In such cases, it has become the norm to resort to back-translating freely available monolingual data as proposed in (Sennrich et al., 2016a). That is, one can trans- late a set of sentences from language L2 into L1 with an already trained MT system for the lan- guage pair L2→L1. Then create a synthetic paral- lel corpus from L1 to L2, with the source (L1) side being the translated text and the target side being the monolingual data. Back-translation has been shown to be beneficial not only for MT but also for other NLP tasks where data is scarce, e.g. auto- matic post-editing (APE) (Junczys-Dowmunt and Grundkiewicz, 2016; Negri et al., 2018). How- ever, the effects of various parameters for creating back-translated (BT) data have not been investi- gated enough as to indicate what are the optimal conditions in not only creating but also employ- ing such data to train high-quality neural machine translation (NMT) systems. The work presented in Poncelas et al. (2018) draws an early-stage empirical roadmap to investi- gating the effects of BT data. In particular, it looks at how the amount of BT data impacts the perfor- mance of the final NMT system. In Sennrich et al. (2016a) and Poncelas et al. (2018), the systems used to generate the BT data are neural. However, it has been noted that often different paradigms can contribute differently to a given task. For ex- ample, it has been shown that applying an APE system based on NMT technology improves sta- tistical machine translation (SMT) output, but has lower impact on NMT output (Bojar …

Content truncated to 500 words