The Karlsruhe Institute of Technology Translation Systems for the WMT 2015

Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 92–97, Lisboa, Portugal, 17-18 September 2015. c⃝2015 Association for Computational Linguistics. The Karlsruhe Institute of Technology Translation Systems for the WMT 2015 Eunah Cho, Thanh-Le Ha, Jan Niehues, Teresa Herrmann, Mohammed Mediani, Yuqi Zhang and Alex Waibel Institute for Anthropomatics and Robotics KIT - Karlsruhe Institute of Technology firstname.lastname@kit.edu Abstract In this paper, the KIT systems submitted to the Shared Translation Task are presented. We participated in two translation direc- tions: from German to English and from English to German. Both translations are generated using phrase-based translation systems. The performance of the systems was boosted by using language models built based on different tokens such as word, part-of-speech, and automacally generated word clusters. The difference in word or- der between German and English is ad- dressed by part-of-speech and syntactic tree-based reordering models. In addition to a discriminative word lexicon, we used hypothesis rescoring using the ListNet al- gorithm after generating the translation with the phrase-based system. We evalu- ated the rescoring using only the baseline features as well as using additional com- putational complex features. 1 Introduction We describe the KIT systems submitted to the Shared Translation Task of the EMNLP 2015 Tenth Workshop on Statistical Machine Transla- tion. They are phrase-based English→German and German→English systems. In order to clean a large amount of noisy web- crawled data, we applied a filtering technique us- ing an SVM classifier. Language models are built based on different tokens, such as word, part- of-speech, and automacally generated word clus- ters. Final systems also include bilingual lan- guage models, part-of-speech and syntactic tree- based reordering models as well as a lexicalized reordering model. For language modeling, a data selection strategy is also applied. A discriminative word lexicon using source context information is used for both translation directions. In this eval- uation campaign we also show that rescoring us- ing the ListNet algorithm improves the translation performance for both directions. This paper is organized as follows. In Section 2, we describe the data we used for training the systems. A detailed description of the systems is given in Section 3. Section 4 shows experimental setups and results along with an analysis. Finally, Section 5 concludes this paper. 2 Data For training data, we use the European Parliament (EPPS), News Commentary (NC) and Common Crawl parallel corpora for both translation direc- tions. For training the language models, we utilize the monolingual target side of the parallel corpora. The News Shuffle data is also used for language modeling. For German→English, we use the Gi- gaword corpus in addition. The systems are optimized on the newstest2013 set and tested on the newstest2014 set. 3 System Description A preprocessing step is applied to the raw data before the actual training. It includes remov- ing excessively long sentences. Sentences with a length mismatch are also filtered out based on a threshold, and special symbols, dates and numbers are normalized. The preprocessing in- cludes smart-casing of the first letter of …

Content truncated to 500 words