Tilde’s Machine Translation Systems for WMT 2017

Proceedings of the Conference on Machine Translation (WMT), Volume 2: Shared Task Papers, pages 374–381 Copenhagen, Denmark, September 711, 2017. c⃝2017 Association for Computational Linguistics Tilde’s Machine Translation Systems for WMT 2017 M¯arcis Pinnis and Rihards Kriˇslauks and Toms Miks and Daiga Deksne and Valters ˇSics Tilde, Vienibas gatve 75A, Riga, Latvia {marcis.pinnis,rihards.krislauks,toms.miks}@tilde.lv {daiga.deksne,valters.sics}@tilde.lv Abstract The paper describes Tilde’s English- Latvian and Latvian-English machine translation systems for the WMT 2017 shared task in news translation. Both con- strained and unconstrained systems are de- scribed. Our constrained systems were ranked as the best performing systems ac- cording to the automatic evaluation re- sults. The paper gives details to how we pre-processed training data, the NMT sys- tem architecture that we used for training the NMT models, the SMT systems and their usage in NMT-SMT hybrid system configurations. 1 Introduction The year 2016 marked the first time when neu- ral machine translation (NMT) systems achieved significantly better results than statistical machine translation (SMT) systems for most of the trans- lation directions in the news translation shared task of the WMT conference. This was achieved due to a number of architectural and data pre- processing novelties that the winning systems in- corporated, for instance, the use of an attention mechanism in the decoder of the NMT system (Bahdanau et al., 2014), back-translation of ad- ditional in-domain monolingual data for domain adaptation of the NMT system after training of a broad domain model or during re-training of the whole NMT model, use of sub-word units to address the problem of out-of-vocabulary word translation, and others (Sennrich et al., 2016). Since then, a number of further advances have been made in machine translation and related fields. A lot of effort has been invested in the search for the best hyper-parameter configurations and neural network architectures for NMT sys- tem training (Britz et al., 2017). In particular, the use of long short-term memory (LSTM) cells and deep architectures has shown to allow increas- ing translation quality. Parallel to that, a num- ber of novelties in neural network architectures have been introduced for other sequence process- ing tasks, some of which, like the multiplica- tive LSTM (MLSTM) units (Krause et al., 2016), promise advantages even over deep recurrent net- work architectures. For data pre-processing, we have shown that the language agnostic word split- ting method using byte pair encoding (BPE) in- consistently splits words for morphologically rich languages and that the method can be improved by linguistically motivating word splitting (Pinnis et al., 2017b). For the WMT 2017 shared task in news trans- lation, we build upon the NMT toolkit Nematus (Sennrich et al., 2016) that achieved the best re- sults in the WMT 2016 shared task. We also in- corporate in our systems the latest advancements in the field, for instance, MLSTM recurrent lay- ers, morphology-driven word splitting, better han- dling of unknown and rare words with robust NMT models, and hybrid methods. The improve- ments over the baseline NMT model have allowed us to develop …

Content truncated to 500 words