KIT’s Multilingual Neural Machine Translation systems for IWSLT 2017

KIT’s Multilingual Neural Machine Translation systems for IWSLT 2017 †Ngoc-Quan Pham, †Matthias Sperber, ⇤†Elizabeth Salesky, †Thanh-Le Ha, †Jan Niehues, ⇤†Alexander Waibel †Institute for Anthropomatics and Robotics KIT - Karlsruhe Institute of Technology, Germany firstname.lastname@kit.edu ⇤ Language Technologies Institute Carnegie Mellon University firstname.lastname@cmu.edu Abstract In this paper, we present KIT’s multilingual neural ma- chine translation (NMT) systems for the IWSLT 2017 eval- uation campaign machine translation (MT) and spoken lan- guage translation (SLT) tasks. For our MT task submissions, we used our multi-task system, modified from a standard attentional neural ma- chine translation framework, instead of building 20 individ- ual NMT systems. We investigated different architectures as well as different data corpora in training such a multilingual system. We also suggested an effective adaptation scheme for multilingual systems which brings great improvements compared to monolingual systems. For the SLT track, in addition to a monolingual neural translation system used to generate correct punctuations and true cases of the data prior to training our multilingual sys- tem, we introduced a noise model in order to make our sys- tem more robust. Results show that our novel modifications improved our systems considerably on all tasks. 1. Introduction In recent works, attention-based neural networks has been considered the state-of-the-art approach for machine trans- lation. More importantly, this framework can be efficiently adapted or customized to fit in a multilingual setting, so that one model can be trained to translate from and to multiple languages. In this evaluation campaign, we empirically ex- plore different architectures which have been exploited in various previous works, in order to find the best combination for the multilingual setting. Specifically, we break down the neural machine transla- tion architecture into its main components: embedding lay- ers, encoders, decoders, attention and output layers. Our analysis indicates which components can be shared to ben- efit from multilingual data. We also employed an adapta- tion strategy which is proved to be beneficial for multi-task learning. Our best systems are the ensembles of individual architectures. 2. Data Processing The data is preprocessed prior to training and translation. Sentence longer than 50 words and aligned sentence pairs having a big difference in length are removed. Special dates, numbers and symbols are normalized. Smartcasing is ap- plied as well. Afterwards, we apply byte pair encoding [1] to model the translation of rare words. We build corpora us- ing 40K codes and 80K codes. Since we did not see a large difference in performance, all reported results use a byte pair encoding size of 40K. 2.1. Sentence alignment While for the in-domain TED corpus, parallel data was pro- vided for all directions, the out-of-domain EPPS data was only available from and to English. For all language direction that do not include English, no additional data was available. In order to generate this data, we used English as a pivot lan- guage, sentence-aligning the English sides of source-English and English-target data in order to extract source-target sen- tence pairs. In the two tracks that we participated in, the …

Content truncated to 500 words