Translating Phrases in Neural Machine Translation

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1421–1431 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Translating Phrases in Neural Machine Translation Xing Wang† Zhaopeng Tu‡ Deyi Xiong†∗ Min Zhang† †Soochow University, Suzhou, China xingwsuda@gmail.com, {dyxiong, minzhang}@suda.edu.cn ‡ Tencent AI Lab, Shenzhen, China tuzhaopeng@gmail.com Abstract Phrases play an important role in natu- ral language understanding and machine translation (Sag et al., 2002; Villavicencio et al., 2005). However, it is difficult to in- tegrate them into current neural machine translation (NMT) which reads and gener- ates sentences word by word. In this work, we propose a method to translate phrases in NMT by integrating a phrase memory storing target phrases from a phrase-based statistical machine translation (SMT) sys- tem into the encoder-decoder architecture of NMT. At each decoding step, the phrase memory is first re-written by the SMT model, which dynamically generates rel- evant target phrases with contextual in- formation provided by the NMT model. Then the proposed model reads the phrase memory to make probability estimations for all phrases in the phrase memory. If phrase generation is carried on, the NMT decoder selects an appropriate phrase from the memory to perform phrase translation and updates its decoding state by con- suming the words in the selected phrase. Otherwise, the NMT decoder generates a word from the vocabulary as the general NMT decoder does. Experiment results on the Chinese→English translation show that the proposed model achieves signif- icant improvements over the baseline on various test sets. 1 Introduction Neural machine translation (NMT) has been re- ceiving increasing attention due to its impressive ∗Corresponding author translation performance (Kalchbrenner and Blun- som, 2013; Cho et al., 2014; Sutskever et al., 2014; Bahdanau et al., 2015; Wu et al., 2016). Sig- nificantly different from conventional statistical machine translation (SMT) (Brown et al., 1993; Koehn et al., 2003; Chiang, 2005), NMT adopts a big neural network to perform the entire trans- lation process in one shot, for which an encoder- decoder architecture is widely used. Specifically, the encoder encodes a source sentence into a con- tinuous vector representation, then the decoder uses the continuous vector representation to gen- erate the corresponding target translation word by word. The word-by-word generation philosophy in NMT makes it difficult to translate multi-word phrases. Phrases, especially multi-word expres- sions, are crucial for natural language understand- ing and machine translation (Sag et al., 2002; Villavicencio et al., 2005) as the meaning of a phrase cannot be always deducible from the mean- ings of its individual words or parts. Unfortu- nately current NMT is essentially a word-based or character-based (Chung et al., 2016; Costa-juss`a and Fonollosa, 2016; Luong and Manning, 2016) translation system where phrases are not consid- ered as translation units. In contrast, phrases are much better than words as translation units in SMT and have made a significant advance in trans- lation quality. Therefore, a natural question arises: Can we translate phrases in NMT? Recently, there have been some attempts on multi-word …

Content truncated to 500 words