The Karlsruhe Institute of Technology Systems for the News Translation Task in WMT 2017

Proceedings of the Conference on Machine Translation (WMT), Volume 2: Shared Task Papers, pages 366–373 Copenhagen, Denmark, September 711, 2017. c⃝2017 Association for Computational Linguistics The Karlsruhe Institute of Technology Systems for the News Translation Task in WMT 2017 Ngoc-Quan Pham, Jan Niehues, Thanh-Le Ha, Eunah Cho, Matthias Sperber, Alexander Waibel Karlsruhe Institute of Technology, Karlsruhe, Germany firstname.lastname@kit.edu Abstract We present our experiments in the scope of the news translation task in WMT 2017, in three directions: German→English, English→German and English→Latvian. The core of our systems is the encoder- decoder based neural machine translation models , enhanced with various modeling features, additional source side augmen- tation and output rescoring. We also ex- periment various methods in data selection and adaptation. 1 Introduction We participate in the WMT 17 shared task on news translation with three directions: English-German, German-English and English-Latvian. The core of our submissions is the neural attentional encoder- decoder model, which we enhanced with different features such as context gates for more efficient attention and the coverage vector for maintaining attentional information during translation. Sev- eral techniques to integrated additional informa- tion into the source text have be investigated: Pre- translation with statistical systems, mono-lingual data and phrase-table entries. Finally, we com- bined different models using n-best lists reranking. 2 Data This section describes the preprocessing steps for the parallel and monolingual corpora for the lan- guage pairs involved in the systems as well as the data selection methods investigated. 2.1 German↔English As parallel data for our German↔English sys- tems, we used Europarl v7 (EPPS), News Com- mentary v12 (NC), Rapid corpus of EU press releases, Common Crawl corpus, and simulated data. Except for the common crawl corpus, no special preprocessing was applied, but only tok- enization and true-casing. For the common crawl corpus, we applied noise filtering using SVM as shown in Mediani et al. (2011). Around 900K sen- tence pairs are filtered out using this technique. Synthetic data is motivated by Sennrich et al. (2015a). In order to exploit the monolingual data, we used the back-translation technique. We ran- domly select sentences from the data as much as our parallel data, and translate them with an in- verse NMT system from the target to the source language. We use this synthetic data as an ad- ditional parallel training data. Summing all cor- pora, the preprocessed and noise-filtered parallel data reaches 8.3M sentences for each language. For German monolingual data, we use News Crawl data. For English, we use News Crawl and News Discussions corpus. Same as for paral- lel data, only tokenization and true-casing are ap- plied. Once the data is preprocessed, we applied byte- pair encoding (BPE) (Sennrich et al., 2015b) on the corpus. In this work, we deploy two different operation sizes, 40K and 80K. 2.1.1 Monolingual data selection We experimented with using domain adaptation techniques to select monolingual data for back- translation. In particular, we concatenated all news-test data sets up until 2013 to form our in- domain corpus, and used news-shuffle as …

Content truncated to 500 words