Adaptation and Combination of NMT Systems: The KIT Translation Systems for IWSLT 2016

Adaptation and Combination of NMT Systems: The KIT Translation Systems for IWSLT 2016 Eunah Cho, Jan Niehues, Thanh-Le Ha, Matthias Sperber, Mohammed Mediani, Alex Waibel Institute for Anthropomatics and Robotics KIT - Karlsruhe Institute of Technology, Germany firstname.lastname@kit.edu Abstract In this paper, we present the KIT systems of the IWSLT 2016 machine translation evaluation. We participated in the ma- chine translation (MT) task as well as the spoken language language translation (SLT) track for English→German and German→English translation. We use attentional neural machine translation (NMT) for all our submissions. We investigated different methods to adapt the system using small in-domain data as well as meth- ods to train the system on these small corpora. In addition, we investigated methods to combine NMT systems that en- code the input as well as the output differently. We combine systems using different vocabularies, reverse translation sys- tems, multi-source translation system. In addition, we used pre-translation systems that facilitate phrase-based machine translation systems. Results show that applying domain adaptation and en- semble technique brings a crucial improvement of 3-4 BLEU points over the baseline system. In addition, system combi- nation using n-best lists yields further 1-2 BLEU points. 1. Introduction The Karlsruhe Institute of Technology participated in the IWSLT 2016 Evaluation Campaign with systems for English→German and German→English. For both direc- tions, we participated in machine translation and spoken language translation tracks. All submitted systems use the framework of attentional neural machine translation [1] ex- tended with further features. In this evaluation campaign, we investigated the impor- tance of domain adaptation, where ensemble technique is deployed for this scenario. In addition to adaptation, we train further systems with different architectures and com- bine them by n-best rescoring. One of the systems uses pre- translation. In this system, we utilize pre-translations from a phrase based machine translation (PBMT) system in order to handle the rare word problem of NMT. The pre-translation is then used as an additional input to the NMT system. Further- more, we used a system utilizing multi-lingual learning. The systems are, along with the ensembled systems of adaptation, combined using the n-best lists. This paper is structured as follows. In Section 2, we de- scribe the adaptation technique we used in order to fit the models better to the domain. A brief explanation of the pre- translation and multi-lingual learning will be given in Section 3 and Section 4 respectively. How the different systems are combined will be described in Section 5. Special preprocess- ing for SLT input will be described afterwards in Section 6, followed by the results of experiments and detailed analy- sis on the techniques used throughout in this work. Finally Section 8 concludes our discussion. 2. Adaptation One of the main challenges of the IWSLT evaluation is to adapt the MT system towards the target domain. While rela- tively large out-of-domain corpora are available for training, the in-domain data is often limited. For the TED task, only around 200K sentences of in-domain data are available. Motivated by the …

Content truncated to 500 words