XMU Neural Machine Translation Systems for WMT 17

Proceedings of the Conference on Machine Translation (WMT), Volume 2: Shared Task Papers, pages 400–404 Copenhagen, Denmark, September 711, 2017. c⃝2017 Association for Computational Linguistics XMU Neural Machine Translation Systems for WMT 17 Zhixing Tan, Boli Wang, Jinming Hu, Yidong Chen and Xiaodong Shi School of Information Science and Engineering, Xiamen University, Fujian, China {playinf, boliwang, todtom}@stu.xmu.edu.cn {ydchen, mandel}@xmu.edu.cn Abstract This paper describes the Neural Machine Translation systems of Xiamen University for the translation tasks of WMT 17. Our systems are based on the Encoder-Decoder framework with attention. We partici- pated in three directions of shared news translation tasks: English→German and Chinese↔English. We experimented with deep architectures, different segmentation models, synthetic training data and target- bidirectional translation models. Experi- ments show that all methods can give sub- stantial improvements. 1 Introduction Neural Machine Translation (NMT) (Cho et al., 2014; Sutskever et al., 2014; Bahdanau et al., 2015) has achieved great success in recent years and obtained state-of-the-art results on various language pairs (Zhou et al., 2016; Sennrich et al., 2016a; Wu et al., 2016). This paper de- scribes the NMT systems of Xiamen University (XMU) for the WMT 17. We participated in three directions of shared news translation tasks: English→German and Chinese↔English. We use two different NMTs for shared news translation tasks: • MININMT: A deep NMT system (Zhou et al., 2016; Wu et al., 2016; Wang et al., 2017) with a simple architecture. The de- coder is a stacked Long Short-Term Mem- ory (LSTM) (Hochreiter and Schmidhuber, 1997) with 8 layers. The encoder has two variants. For English-German translation, we use an interleaved bidirectional encoder with 2 columns. Each column consists of 4 LSTMs. For Chinese-English translation, we use a stacked bidirectional encoder with 8 layers. • DL4MT: Our reimplementation of dl4mt- tutorial1 with minor changes. We also use a modified version of AmuNMT C++ decoder2 for decoding. This system is used in the English-Chinese translation task. We use both Byte Pair Encoding (BPE) (Sennrich et al., 2016c) and mixed word/character segmenta- tion (Wu et al., 2016) to achieve open-vocabulary translation. Back-translation method (Sennrich et al., 2016b) is applied to make use of monolin- gual data. We also use target-bidiretional trans- lation models to alleviate the label bias prob- lem (Lafferty et al., 2001). The remainder of this paper is organized as fol- lows: Section 2 describes the architecture of MIN- INMT. Section 3 describes all experimental fea- tures used in WMT 17 shared translation tasks. Section 4 shows the results of our experiments. Section 5 shows the results of shared translation task. Finally, we conclude in section 6. 2 Model Description Deep architectures have recently shown promis- ing results on various language pairs (Zhou et al., 2016; Wu et al., 2016; Wang et al., 2017). We also experimented with a deep architecture as depicted in Figure 1. We use LSTM as the main recurrent unit and residual connections (He et al., 2016) to help training. Given a source sentence x = {x1, . . . , xS} and …

Content truncated to 500 words