Kyoto University MT System Description for IWSLT 2017

Kyoto University MT System Description for IWSLT 2017 Raj Dabre1, Fabien Cromieres2 and Sadao Kurohashi1 1 Graduate School of Informatics, Kyoto University, Kyoto, Japan 2 Japan Science and Technology Agency, Saitama, Japan 1 dabre@nlp.ist.i.kyoto-u.ac.jp 2 fabien@nlp.ist.i.kyoto-u.ac.jp 1 kuro@i.kyoto-u.ac.jp Abstract We describe here our Machine Translation (MT) model and the results we obtained for the IWSLT 2017 Multilingual Shared Task. Motivated by Zero Shot NMT [1] we trained a Multilingual Neural Machine Translation by combining all the training data into one single collection by appending the tokens: ”< 2xx >” (where xx is the language code of the target language) to the source sentences in order to indicate the target language they should be translated to. We ob- served that even in a low resource situation we were able to get translations whose quality surpass the quality of those obtained by Phrase Based Statistical Machine Translation by several BLEU points. The most surprising result we obtained was in the zero shot setting for Dutch-German and Italian- Romanian where we observed that despite using no parallel corpora between these language pairs, the NMT model was able to translate between these languages and the translations were either as good as or better (in terms of BLEU) than the non zero resource setting. We also verify that the NMT mod- els that use feed forward layers and self attention instead of recurrent layers are extremely fast in terms of training which is useful in a NMT experimental setting. 1. Introduction One of the most attractive features of neural machine transla- tion (NMT) [2, 3, 4] is that it is possible to train an end to end system without the need to deal with word alignments, trans- lation rules and complicated decoding algorithms, which are a characteristic of statistical machine translation (SMT) sys- tems [5]. However, it is reported that NMT works better than SMT only when there is an abundance of parallel corpora. In the case of low resource domains, vanilla NMT is either worse than or comparable to SMT, due to overfitting on the small size of parallel corpora [6]. Although PBSMT is superior to NMT in low resource situations it leads to large models (phrase and reordering ta- bles and language models) and thus is not an attractive ap- proach, especially because it cannot lead to the development of models that are end to end. Recently, Google’s multilin- gual system was made available to the public which was able to perform Zero Shot translation [1]. Although, it is possi- ble to train a multilingual NMT model using a multi encoder and decoder setup [7], such a model contains a massive num- ber of parameters and does not enable interaction between languages by means of shared encoders and decoders. More- over, it is clear that the basic attention based encoder-decoder model is more than capable of accommodating multiple lan- guages while keeping the number of parameters constant. Multilingual NMT (MLNMT) models are inherently more powerful than bilingual models especially when the target language …

Content truncated to 500 words