Enabling Multi-Source Neural Machine Translation By Concatenating Source Sentences In Multiple Languages

Enabling Multi-Source Neural Machine Translation By Concatenating Source Sentences In Multiple Languages Raj Dabre raj@nlp.ist.i.kyoto-u.ac.jp Graduate School of Informatics, Kyoto University, Kyoto, Japan Fabien Cromieres fabien@pa.jst.jp Japan Science and Technology Agency, Saitama, Japan Sadao Kurohashi kuro@i.kyoto-u.ac.jp Graduate School of Informatics, Kyoto University, Kyoto, Japan Abstract In this paper, we explore a simple solution to “Multi-Source Neural Machine Translation" (MSNMT) which only relies on preprocessing a N-way multilingual corpus without modifying the Neural Machine Translation (NMT) architecture or training procedure. We simply concate- nate the source sentences to form a single long multi-source input sentence while keeping the target side sentence as it is and train an NMT system using this preprocessed corpus. We eval- uate our method in resource poor as well as resource rich settings and show its effectiveness (up to 4 BLEU using 2 source languages and up to 6 BLEU using 5 source languages). We also compare against existing methods for MSNMT and show that our solution gives competitive results despite its simplicity. We also provide some insights on how the NMT system leverages multilingual information in such a scenario by visualizing attention. 1 Introduction Multi-Source Machine Translation (MSMT) Och and Ney (2001) is an approach that allows one to leverage source sentences in multiple languages to improve the translations to a target lan- guage. Typically N-way (or N-lingual) corpora are used for MSMT. N-way corpora are those in which translations of the same sentence exist in N different languages. This setting is realistic and has many applications. For example, the European Parliament maintains its proceedings in 21 languages. In Spain, international news companies write news articles in English as well as Spanish. One can now utilize the same sentence written in two different languages like Spanish and English to translate to a third language like Italian by utilizing a large English-Spanish- Italian trilingual corpus. Neural machine translation (NMT) Bahdanau et al. (2015); Cho et al. (2014); Sutskever et al. (2014) enables one to train an end-to-end system without the need to deal with word align- ments, translation rules and complicated decoding algorithms, which are a characteristic of phrase based statistical machine translation (PBSMT) systems. However, it is reported that NMT works better than PBSMT only when there is an abundance of parallel corpora. In a low resource scenario, vanilla NMT is either worse than or comparable to PBSMT Zoph et al. (2016). Multi-source Neural Machine translation (MSNMT) involves using NMT for MSMT. Two Proceedings of MT Summit XVI, vol.1: Research Track Nagoya, Sep. 18-22, 2017 | p. 96 major approaches for Multi-Source NMT (MSNMT) have been explored, namely the multi- encoder (ME/me) Zoph and Knight (2016) and multi-source ensembling (ENS/ens) Garmash and Monz (2016); Firat et al. (2016). The multi-encoder approach involves extending the vanilla NMT architecture to have an encoder for each source language leading to larger models. On the other hand, the ensembling approach is simpler since it involves training multiple bilingual NMT models each with a different source language but the same target …

Content truncated to 500 words