Neural Machine Translation Leveraging Phrase-based Models in a Hybrid Search

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1411–1420 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Neural Machine Translation Leveraging Phrase-based Models in a Hybrid Search Leonard Dahlmann and Evgeny Matusov and Pavel Petrushkov and Shahram Khadivi eBay Inc. Kasernenstr. 25 52064 Aachen, Germany {fdahlmann, ematusov, ppetrushkov, skhadivi}@ebay.com Abstract In this paper, we introduce a hybrid search for attention-based neural machine trans- lation (NMT). A target phrase learned with statistical MT models extends a hypoth- esis in the NMT beam search when the attention of the NMT model focuses on the source words translated by this phrase. Phrases added in this way are scored with the NMT model, but also with SMT features including phrase-level translation probabilities and a target language model. Experimental results on German→English news domain and English→Russian e- commerce domain translation tasks show that using phrase-based models in NMT search improves MT quality by up to 2.3% BLEU absolute as compared to a strong NMT baseline. 1 Introduction Neural machine translation has become state-of- the-art in recent years, reaching higher transla- tion quality than statistical phrase-based machine translation (PBMT) on many tasks. Human anal- ysis (Bentivogli et al., 2016) showed that NMT makes significantly fewer reordering errors, and also is able to select correct word forms more of- ten than PBMT in the case of morphologically rich target languages. Overall, the fluency of the MT output improves when NMT is used, and the number of lexical choice errors is also reduced. However, state-of-the-art NMT approaches based on an encoder-decoder architecture with an atten- tion mechanism as introduced by (Bahdanau et al., 2014) exhibit weaknesses that sometimes lead to MT errors which a phrase-based MT system does not make. In particular, PBMT usually can better translate rare words (e.g. singletons), as well as memorize and use phrasal translations. NMT has problems translating rare words because of limi- tations on the vocabulary size, as well as the fact that word embeddings are used to represent both source and target words. A rare word’s embedding can not be trained reliably. Another handicap of NMT is a general diffi- culty of fixing errors made by a neural MT sys- tem. Since NMT does not explicitly use or save word-to-word or phrase-to-phrase mappings, and its search is a target word beam search with al- most no constraints, it is difficult to fix errors by an NMT system. It is important to quickly fix cer- tain errors in real-life applications of MT systems to avoid negative user feedback or other (e.g. le- gal) consequences. An error identified in the out- put of a PBMT system can be fixed by tracing which phrase pair was used that resulted in the error, and down-weighting or even removing the phrase pair. Also, in PBMT it is easy to add an “override” translation. In this work, we combine the strengths of NMT and PBMT approaches by introducing a novel hy- brid search algorithm. In this algorithm, the stan- dard …

Content truncated to 500 words