FBK’s Multilingual Neural Machine Translation System for IWSLT 2017

FBK’s Multilingual Neural Machine Translation System for IWSLT 2017 Surafel M. Lakew1,2, Quintino F. Lotito2, Marco Turchi1, Matteo Negri1, Marcello Federico1 1Fondazione Bruno Kessler, Trento, Italy 2University of Trento, Trento, Italy federico@fbk.eu Abstract Neural Machine Translation has been shown to enable in- ference and cross-lingual knowledge transfer across multi- ple language directions using a single multilingual model. Focusing on this multilingual translation scenario, this work summarizes FBK’s participation in the IWSLT 2017 shared task. Our submissions rely on two multilingual systems trained on five languages (English, Dutch, German, Ital- ian, and Romanian). The first one is a 20 language direc- tion model, which handles all possible combinations of the five languages. The second multilingual system is trained only on 16 directions, leaving the others as zero-shot trans- lation directions (i.e representing a more complex inference task on language pairs not seen at training time). More specifically, our zero-shot directions are Dutch$German and Italian$Romanian (resulting in four language combi- nations). Despite the small amount of parallel data used for training these systems, the resulting multilingual models are effective, even in comparison with models trained sepa- rately for every language pair (i.e. in more favorable condi- tions). We compare and show the results of the two multi- lingual models against a baseline single language pair sys- tems. Particularly, we focus on the four zero-shot directions and show how a multilingual model trained with small data can provide reasonable results. Furthermore, we investigate how pivoting (i.e using a bridge/pivot language for inference in a source!pivot!target translations) using a multilingual model can be an alternative to enable zero-shot translation in a low resource setting. 1. Introduction Recently, multilingual translation across different languages using a single model showed to perform in a comparable way with single language pair systems. In [1, 2], a multilingual model has been successfully trained using a standard Neu- ral Machine Translation (NMT) architecture by applying a simple preprocessing step on the source side of the training data. It consists in prepending an artificial language token indicating the target language id at the beginning of each sentence. This information guides the system towards a spe- cific target language both at training and inference time. This mechanism of guiding the multilingual model is referred to as target-forcing [2]. Multilingual NMT English Italian Romanian English Italian Romanian Dutch Dutch German German Figure 1: The multilingual system source!target associ- ation. A parallel data exists for all the 20 directions in the first multilingual model, where as zero-shot model the Dutch$German and Italian$Romanian pairs (dashed line) are excluded In this work, we present our participation in two IWSLT20171 [3] shared tasks: i) a multilingual translation task in a small data condition for twenty language directions, and ii) a multilingual zero-shot task in a similar small data condition. For convenience, throughout the paper, we refer to the models trained for the two tasks respectively as Multilin- gual and Zero-shot models. We trained the two models sep- arately, by sharing a common configuration. The only differ- ence, …

Content truncated to 500 words