Unsupervised Extraction of Partial Translations for Neural Machine Translation

Proceedings of NAACL-HLT 2019, pages 3834–3844 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 3834 Unsupervised Extraction of Partial Translations for Neural Machine Translation Benjamin Marie Atsushi Fujita National Institute of Information and Communications Technology 3-5 Hikaridai, Seika-cho, Soraku-gun, Kyoto, 619-0289, Japan {bmarie, atsushi.fujita}@nict.go.jp Abstract In neural machine translation (NMT), mono- lingual data are usually exploited through a so- called back-translation: sentences in the tar- get language are translated into the source lan- guage to synthesize new parallel data. While this method provides more training data to bet- ter model the target language, on the source side, it only exploits translations that the NMT system is already able to generate using a model trained on existing parallel data. In this work, we assume that new translation knowl- edge can be extracted from monolingual data, without relying at all on existing parallel data. We propose a new algorithm for extracting from monolingual data what we call partial translations: pairs of source and target sen- tences that contain sequences of tokens that are translations of each other. Our algorithm is fully unsupervised and takes only source and target monolingual data as input. Our empirical evaluation points out that our par- tial translations can be used in combination with back-translation to further improve NMT models. Furthermore, while partial transla- tions are particularly useful for low-resource language pairs, they can also be successfully exploited in resource-rich scenarios to im- prove translation quality. 1 Introduction Neural machine translation (NMT) systems usu- ally require a large quantity of high-quality bilin- gual parallel data for training. However, for most language pairs, we do not have such resources, or only in very small quantities, mainly because they are costly to produce. On the other hand, monolingual corpora are readily available in large quantity for many lan- guages. Previous work has proposed various strategies to integrate monolingual data into NMT systems and has confirmed their usefulness to improve NMT systems. The so-called back- translation of monolingual data (Sennrich et al., 2016) is undoubtedly the most prevalent one. This approach simply uses a target-to-source MT sys- tem to translate monolingual data in the target lan- guage into the source language. The produced new synthetic parallel corpus can be used together with the original parallel data to increase the size of the training data, and eventually to improve NMT systems significantly and consistently. However, on the source side, the synthetic data only contain data that can be generated by the back-translation system trained on some existing parallel data. Previous work has also studied the extraction of translation pairs of source and target sentences from monolingual data in their respective lan- guages. They have been shown to be useful to train better statistical machine translation (SMT) sys- tems, especially in low-resource conditions. Ex- isting methods on sentence pair extraction mainly rely on the availability of comparable corpora as the source of accurate sentence pairs (Abdul Rauf and Schwenk, 2011), or on the robustness of SMT against noise …

Content truncated to 500 words