CASICT-DCU Neural Machine Translation Systems for WMT17

Proceedings of the Conference on Machine Translation (WMT), Volume 2: Shared Task Papers, pages 428–431 Copenhagen, Denmark, September 711, 2017. c⃝2017 Association for Computational Linguistics CASICT-DCU Neural Machine Translation Systems for WMT17 Jinchao Zhang1 Peerachet Porkaew1 Jiawei Hu1 Qiuye Zhao1 Qun Liu2,1 1Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences 2ADAPT Centre, School of Computing, Dublin City University {zhangjinchao,dingchunfa,hujiawei,zhaoqiuye,liuqun}@ict.ac.cn Abstract We participated in the WMT 2016 shared news translation task on English ↔ Chi- nese language pair. Our systems are based on the encoder-decoder neural machine translation model with the attention mech- anism. We employ the Gated Recurrent Unit (GRU) with the linear associative connection to build deep encoder and ad- dress the unknown words with the dic- tionary replace approach. The dictionar- ies are extracted from the parallel train- ing data with unsupervised word align- ment method. In the decoding procedure, the translation probabilities of the target word from different models are averagely combined as the ensemble strategy. In this paper, we introduce our systems from data preprocessing to post-editing in details. 1 Introduction We build the Neural Machine Translation sys- tems CASICT-DCU for WMT17 English ↔ Chi- nese news translation task. Our systems are based on the encoder-decoder model with the at- tention mechanism, which is also known as the RNNSearch model (Bahdanau et al., 2015). To construct the deep RNN network, we employ the Gated Recurrent Unit (Cho et al., 2014b) with the linear associative connection (Wang et al., 2017) to ensure the fluent gradient propagation. Adadelta (Zeiler, 2012) algorithm is used to op- timize the parameters and stochastic gradient de- scent algorithm with small learning rate is used in the fine-tuning stage. We extract dictionaries from parallel training data with the unsupervised method to address the unknown words in target translation according to the word alignment vec- tor. During the decoding, the ensemble strategy is used to combine the translation probabilities of the target word from different models. 2 System Description The neural machine translation model (Kalch- brenner and Blunsom, 2013; Cho et al., 2014b; Sutskever et al., 2014) aims to capture the transla- tion knowledge through training a neural network in the end-to-end style. Our systems are built on the RNNSearch neural machine translation model. Formally, given a source sentence x = x1, ..., xm and a target sentence y = y1, ..., yn, NMT models the translation probability as P(y|x) = n � t=1 P(yt|y<t, x), (1) where y<t = y1, ..., yt−1. The generation proba- bility of yt is P(yt|y<t, x) = g(yt−1, ct, st), (2) where g(·) is a softmax regression function, yt−1 is the newly translated target word and st is the hidden states of decoder which represents the translation status. The attention ct denotes the re- lated source words for generating yt and is com- puted as the weighted-sum of source represen- tation h upon an alignment vector αt shown in Eq.(3) where the align(·) function is a feedfor- ward network with softmax normalization. …

Content truncated to 500 words