The RWTH Aachen Machine Translation Systems for IWSLT 2017

The RWTH Aachen Machine Translation Systems for IWSLT 2017 Parnia Bahar⇤, Jan Rosendahl⇤, Nick Rossenbach and Hermann Ney Human Language Technology and Pattern Recognition Group Computer Science Department RWTH Aachen University Aachen, Germany <surname>@cs.rwth-aachen.de ⇤Authors contributed equally Abstract This work describes the Neural Machine Translation (NMT) system of the RWTH Aachen University developed for the English$German tracks of the evaluation campaign of the International Workshop on Spoken Language Translation (IWSLT) 2017. We use NMT systems which are augmented by state-of-the-art extensions. Furthermore, we experiment with techniques that include data filtering, a larger vocabu- lary, two extensions to the attention mechanism and domain adaptation. Using these methods, we can show considerable improvements over the respective baseline systems and our IWSLT 2016 submission. 1. Introduction We describe the Neural Machine Translation (NMT) system of the RWTH Aachen University developed for the evalu- ation campaign of International Workshop on Spoken Lan- guage Translation (IWSLT) 2017. We have participated in the unofficial bilingual Machine Translation (MT) track for the German!English and English!German language pairs. The in-house NMT system incorporates various state-of-the- art extensions. For the IWSLT 2016 evaluation campaign, RWTH Aachen utilized different translation systems [1] including a state-of-the-art phrase-based system, a neural machine trans- lation system and the joint translation and reordering (JTR) model [2]. Furthermore, last year’s system applied feed- forward and recurrent neural language and translation mod- els for reranking. The attention-based approach had been used for reranking the n-best lists for both the phrase-based and the hierarchical setups. On top of these systems, a system combination enhances the translation quality by combining individually trained systems. For the IWSLT 2017 evalua- tion campaign, we developed the systems only based on the NMT approach as it has shown the most promising results among all. This paper is organized as follows. In Section 2, we briefly address our preprocessing which differs from our pre- vious submissions [3, 1]. Section 3 describes the details of the NMT systems, the baseline, our optimization techniques as well as two extensions to the attention mechanism. Our experiments for each track are summarized in Section 4. 2. Preprocessing 2.1. Preprocessing Recent studies [4] showed that attention-based neural net- work systems do not benefit from several established prepro- cessing features such as compound splitting and POS-based word reordering. Therefore, we decided to employ a sim- pler version of preprocessing which uses only tokenization, frequent casing, and simple categories. In this approach, numbers are not mapped to a specific category-token but are treated like regular words instead. All words and numbers are split into subword units us- ing byte-per-encoding (BPE)1 introduced by [5]. We use 90k BPE merging operations trained jointly on the concatenated source and target training data. In the preprocessing, we do not distinguish if a language is seen as a source or target lan- guage. 2.2. Data Filtering In order to remove incorrectly aligned sentence pairs, we drop all training samples for which the length of the source sentence exceeds the length of the target sentence by …

Content truncated to 500 words