The RWTH Aachen University English-German and German-English Machine Translation System for WMT 2017

Proceedings of the Conference on Machine Translation (WMT), Volume 2: Shared Task Papers, pages 358–365 Copenhagen, Denmark, September 711, 2017. c⃝2017 Association for Computational Linguistics The RWTH Aachen University English-German and German-English Machine Translation System for WMT 2017 Jan-Thorsten Peter, Andreas Guta, Tamer Alkhouli, Parnia Bahar, Jan Rosendahl, Nick Rossenbach, Miguel Grac¸a and Hermann Ney Human Language Technology and Pattern Recognition Group Computer Science Department RWTH Aachen University D-52056 Aachen, Germany <surname>@i6.informatik.rwth-aachen.de Abstract This paper describes the statistical machine translation system developed at RWTH Aachen University for the English→German and German→English translation tasks of the EMNLP 2017 Second Conference on Machine Transla- tion (WMT 2017). We use ensembles of attention-based neural machine translation system for both directions. We use the provided parallel and synthetic data to train the models. In addition, we also cre- ate a phrasal system using joint translation and reordering models in decoding and neural models in rescoring. 1 Introduction We describe the statistical machine transla- tion (SMT) systems developed by RWTH Aachen University for the German→English and English→German language pairs of the WMT 2017 evaluation campaign. After testing multiple systems and system combinations we submitted an ensemble of multiple NMT networks since it outperformed every tested system combination. This paper is organized as follows. In Section 2 we describe our data preprocessing. Section 3 de- picts the generation of synthetic data. Our transla- tion software and baseline setups are explained in Section 4, including the attention-based recurrent neural network ensemble in Subsection 4.1 and phrasal joint translation and reordering (JTR) sys- tem in Subsection 4.2. Our experiments for each track are summarized in Section 5. 2 Preprocessing We compared two different preprocessings for German→English for the attention-based recur- rent neural network (NMT) system. The first pre- processing is similar to the preprocessing used in our WMT 2015 submission (Peter et al., 2015), which was optimized for phrase-based translation (PBT). Secondly, we utilize a simplified version which uses tokenization, frequent casing, and simple cat- egories only. Note, that the changes in prepro- cessing have a huge negative impact on the PBT system, while slightly improving the NMT system (Table 1). We therefore use the simplified version for all pure NMT experiments and use the old pre- processing for all other systems. The phrasal JTR system uses the preprocess- ing technique that is optimized for PBT, as it re- lies on phrases as translation candidates. The pre- processing is similar to the one used in the WMT 2015 submission, but without any pre-ordering of source words. The English→German NMT sys- tem utilizes only the simplified preprocessing. 3 Synthetic Source Sentences To increase the amount of usable parallel train- ing data for the phrase-based and the neural ma- chine translation systems, we translate a subset of the monolingual training data back to English in a similar way as described by (Bertoldi and Fed- erico, 2009) and (Sennrich et al., 2016b). We create a baseline German→English NMT system as described in 4.1 which is trained with all parallel data to …

Content truncated to 500 words