Sogou Neural Machine Translation Systems for WMT17

Proceedings of the Conference on Machine Translation (WMT), Volume 2: Shared Task Papers, pages 410–415 Copenhagen, Denmark, September 711, 2017. c⃝2017 Association for Computational Linguistics 1 Sogou Neural Machine Translation Systems for WMT17 Yuguang Wang*, Xiang Li*†, Shanbo Cheng*, Liyang Jiang*, Jiajun Yang* Wei Chen*, Lin Shi*, Yanfeng Wang*, Hongtao Yang* *Voice Interaction Technology Center, Sogou Inc., Beijing, China †Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences {wangyuguang, chengshanbo, chenweibj8871}@sogou-inc.com, lixiang@ict.ac.cn Abstract We describe the Sogou neural machine translation systems for the WMT 2017 Chinese↔English news translation tasks. Our systems are based on a multi- layer encoder-decoder architecture with attention mechanism. The best translation is obtained with ensemble and reranking techniques. We also pro- pose an approach to improve the named entity translation problem. Our Chi- nese→English system achieved the highest cased BLEU among all 20 sub- mitted systems, and our English→Chi- nese system ranked the third out of 16 submitted systems.1 1 Introduction End-to-end neural machine translation (NMT) has recently been introduced as a promising paradigm with the potential to address many shortcomings of traditional statistical machine translation (SMT) systems, and has obtained state-of-the-art perfor- mance for several language pairs (Cho et al., 2014; Sutskever et al., 2014; Bahdanau et al., 2015; Sennrich et al., 2016a; Wu et al., 2016; Zhou et al., 2016). In this paper, we describe the Sogou NMT systems submissions for the WMT 2017 Chi- nese→English and English→Chinese translation tasks. Overview of the systems can be described as fol- lows: we implement a multi-layer attention-based encoder-decoder integrated with recent promising techniques in NMT, including that we use subword units based on byte pair encoding (BPE) rather than words as modelling units (Sennrich et al., 2016b) and layer normalization (Ba et al., 2016) to isolated layers. And we improve the performance using en- semble based four systems of the same network 1 Automatic rankings are from http://matrix.statmt.org. trained with different random seeds of parameter initialization. In addition, we improve the performance further by reranking the n-best translation lists with some effective features, including the target-bidirectional models, target-to-source models, and n-gram lan- guage models. And we use another NMT model to translate the recognized person names for the Chi- nese→English task, in order to improve the perfor- mance of unknown named entity translation. Our Chinese→English system achieved the highest cased BLEU among all 20 submitted sys- tems, and our English→Chinese system ranked the third out of 16 submitted systems. 2 Neural Machine Translation Our NMT model follows the common attentional encoder-decoder networks (Bahdanau et al., 2015). We implement a deep multi-layer Long Short Term Memory (LSTM) recurrent neural network for both the encoder and decoder. In our setup, the encoder has one bi-directional LSTM layer followed by two uni-directional LSTM layers. The decoder has three uni-directional LSTM layers. Similar to the conditional GRU used in DL4MT (Firat and Cho, 2016), we use conditional LSTM (cLSTM) for the top layer of decoder instead of standard LSTM. The encoder takes the model’s input …

Content truncated to 500 words