SYSTRAN Purely Neural MT Engines for WMT2017

Proceedings of the Conference on Machine Translation (WMT), Volume 2: Shared Task Papers, pages 265–270 Copenhagen, Denmark, September 711, 2017. c⃝2017 Association for Computational Linguistics SYSTRAN Purely Neural MT Engines for WMT2017 Yongchao Deng, Jungi Kim, Guillaume Klein, Catherine Kobus, Natalia Segal, Christophe Servan, Bo Wang, Dakun Zhang, Josep Crego, Jean Senellart firstname.lastname@systrangroup.com SYSTRAN / 5 rue Feydeau, 75002 Paris, France Abstract This paper describes SYSTRAN’s sys- tems submitted to the WMT 2017 shared news translation task for English-German, in both translation directions. Our sys- tems are built using OpenNMT1, an open- source neural machine translation system, implementing sequence-to-sequence mod- els with LSTM encoder/decoders and at- tention. We experimented using mono- lingual data automatically back-translated. Our resulting models are further hyper- specialised with an adaptation technique that finely tunes models according to the evaluation test sentences. 1 Introduction We participated in the WMT 2017 shared news translation task on two different translation direc- tions: English→German and German→English. The paper is structured as follows: Section 2 overviews our neural MT engine. Section 3 de- scribes the set of experiments carried out to build the English→German and German→English neu- ral translation models. Experiments and results are detailed in Section 3. Finally, conclusions are drawn in Section 4. 2 Neural MT System Neural machine translation (NMT) is a new methodology for machine translation that has led to remarkable improvements, particularly in terms of human evaluation, compared to rule-based and statistical machine translation (SMT) systems (Crego et al., 2016; Wu et al., 2016). NMT has now become a widely-applied technique for ma- chine translation, as well as an effective approach 1http://opennmt.net for other related NLP tasks such as dialogue, pars- ing, and summarisation. Our NMT system (Klein et al., 2017) follows the architecture presented in (Bahdanau et al., 2014). It is implemented as an encoder-decoder network with multiple layers of a RNN with Long Short-Term Memory (LSTM) hidden units (Zaremba et al., 2014). Figure 1 illustrates an schematic view of the MT network. Source words are first mapped to word vectors and then fed into a bidirectional recurrent neu- ral network (RNN) that reads an input sequence s = (s1, ..., sJ). Upon seeing the <eos> sym- bol, the final time step initialises a target RNN. The decoder is a RNN that predicts a target se- quence t = (t1, ..., tI), being J and I respectively the source and target sentence lengths. Translation is finished when the decoder predicts the <eos> symbol. The left-hand side of the figure illustrates the bidirectional encoder, which actually consists of two independent LSTM encoders: one encoding the normal sequence (solid lines) that calculates a forward sequence of hidden states (−→ h1, ..., −→ hJ), the second encoder reads the input sequence in re- versed order (dotted lines) and calculates the back- ward sequence (←− h1, ..., ←− hJ). The final encoder out- puts (h1, ..., hJ) consist of the sum of both en- coders final outputs. The right-hand side of the figure illustrates the RNN decoder. Each …

Content truncated to 500 words