The Helsinki Neural Machine Translation System

Proceedings of the Conference on Machine Translation (WMT), Volume 2: Shared Task Papers, pages 338–347 Copenhagen, Denmark, September 711, 2017. c⃝2017 Association for Computational Linguistics The Helsinki Neural Machine Translation System∗ Robert ¨Ostling Department of Linguistics Stockholm University Yves Scherrer and J¨org Tiedemann Department of Modern Languages University of Helsinki Gongbo Tang Department of Linguistics and Philology Uppsala University Tommi Nieminen Department of Modern Languages University of Helsinki Abstract We introduce the Helsinki Neural Machine Translation system (HNMT) and how it is applied in the news translation task at WMT 2017, where it ranked first in both the human and automatic evaluations for English–Finnish. We discuss the success of English–Finnish translations and the overall advantage of NMT over a strong SMT baseline. We also discuss our sub- missions for English–Latvian, English– Chinese and Chinese–English. 1 Introduction The Helsinki Neural Machine Translation system (HNMT) is a full-featured system for neural ma- chine translation, with a particular focus on mor- phologically rich languages. We participated in the WMT 2017 shared task on news translation, obtaining the highest BLEU score for English– Finnish translation, while also performing well on English–Latvian and acceptably on English– Chinese and Chinese–English. In addition to our participation in the shared task, this paper also details some of the other methods we have implemented and evaluated with HNMT, many of which yielded negative results and were subsequently not used in our submis- sions for the shared task. 1.1 HNMT HNMT is based on the attentional encoder– decoder model due to Bahdanau et al. (2014). This is a rather minimalistic framework for NMT, and many extensions have been proposed. Of particu- lar interest are those that allow proper and efficient ∗The software is available from https://github. com/robertostling/hnmt under the GNU General Public License version 3. handling of morphologically rich languages, such as Finnish. We combine two such approaches: the hybrid character/word model of Luong and Man- ning (2016), which is used for the source language encoder, and the byte-pair encoding (BPE) tech- nique of Sennrich et al. (2016c), which is used for the target language decoder and has been suc- cessfully used for Finnish previously (S´anchez- Cartagena and Toral, 2016). As BPE can be added as a simple pre- and post-processing step, it does not affect the structure of the translation model. This means that our system can be used with char- acter, word and BPE level generation on the tar- get side. The structure of the network, thus, con- sists of three Long Short-Term Memory (LSTM) (Hochreiter and Schmidhuber, 1997) layers: 1. A character-level encoder that transforms out-of-vocabulary source tokens into the same vector space as the source word embed- dings. 2. A token-level bidirectional encoder that transforms a sequence of source word em- beddings (or outputs from network (1) in case of OOV items) into an encoded sequence of the same length. 3. A character-, token- or BPE-level decoder that works as language model conditioned (via an attention mechanism) on the encoded source sequence from (2). HNMT is …

Content truncated to 500 words