The Samsung and University of Edinburgh’s submission to IWSLT17

The Samsung and University of Edinburgh’s submission to IWSLT17 Pawel Przybysz1, Marcin Chochowski1, Rico Sennrich2, Barry Haddow2 and Alexandra Birch2 1Samsung R&D Institute Poland 2School of Informatics, University of Edinburgh {m.chochowski,p.przybysz}@samsung.com bhaddow@inf.ed.ac.uk, {rico.sennrich,a.birch}@ed.ac.uk Abstract This paper describes the joint submission of Samsung Research and Development, Warsaw, Poland and the University of Edinburgh team to the IWSLT MT task for TED talks. We took part in two translation directions, en-de and de-en. We also participated in the en-de and de-en lectures SLT task. The models have been trained with an attentional encoder-decoder model using the BiDeep model in Nematus. We filtered the training data to reduce the problem of noisy data, and we use back-translated monolingual data for domain-adaptation. We demonstrate the effectiveness of the different techniques that we applied via ablation studies. Our submission system outperforms our baseline, and last year’s University of Edinburgh submission to IWSLT, by more than 5 BLEU. 1. Introduction This paper describes the system submission of Sam- sung R&D Institute Poland and the University of Edinburgh team. The models have been trained with a deep attentional encoder-decoder neural machine translation model using Nematus [1]. In this year’s sub- mission, we focused on the core NMT architecture, for which we selected the BiDeep model by [2], training data filtering via language identification and MT-based sentence alignment scores, and domain adaptation with back-translated, domain-filtered monolingual training data, and fine-tuning towards the in-domain training set with MAP-L2 regularization towards the baseline model [3]. Corpus raw aligned filtered Commoncrawl [4] 2.40M 2.22M 1.62M Europarl v7 [5] 1.92M 1.90M 1.85M GoldAligment 509 508 486 MultiUN [6] 0.16M 0.16M 0.15M News Com. v12 [4] 0.27M 0.26M 0.26M Opensubtitles2016 [7] 13.88M 12.08M 9.04M QED Corpus 0.07M 0.07M 0.06M Rapid 2016 1.33M 1.28M 1.12M Wikipedia Corpus 2.46M 2.16M 1.18M WIT3 (in-domain) [8] 0.22M 0.21M 0.20M Total 22.72M 20.35M 15.47M Table 1: Admissible parallel corpora used for training, with number of sentences before and after filtering 2. Training data and data selection 2.1. Parallel corpora For the English-German language pair, we used the cor- pora listed in Table 1. IWSLT provides a large amount of permissible parallel training data. We performed fil- tering based on sentence alignment and language iden- tification. To obtain a sentence alignment score, we follow the idea that we can automatically translate the source text, and use BLEU between the automatic translation and the target side as a feature to predict probable alignments [9]. We trained a Phrase-based Statistical MT model, using significance filtering [10] to remove improbable phrases. Then we translated German sen- tences into English with a fast Statistical MT engine. Then, a sentence aligner BLEU-Champ1 was applied to score each parallel training sentence. We also scored each sentence pair with a sentence-level language 1https://github.com/emjotde/bleu-champ _____________________________________________________________ Proceedings of the 14th International Workshop on Spoken Language Translation Tokyo, Japan, December 14th-15th, 2017 23 recognition tool. After these operations each sentence pair had assigned BLEU-Champ scores and language recognition scores. We selected small subset of 3k sentences from the corpora …

Content truncated to 500 words