Guided Alignment Training for Topic-Aware Neural Machine Translation

Guided Alignment Training for Topic-Aware Neural Machine Translation Wenhu Chen hustchenwenhu@gmail.com RWTH Aachen University, Ahornstr. 55, Aachen, Germany Evgeny Matusov ematusov@ebay.com eBay, Inc. Kasernenstr. 25, Aachen, Germany Shahram Khadivi skhadivi@ebay.com eBay, Inc. Kasernenstr. 25, Aachen, Germany Jan-Thorsten Peter peter@cs.rwth-aachen.de RWTH Aachen University, Ahornstr. 55 Aachen, Germany Abstract In this paper, we propose an effective way for biasing the attention mechanism of a sequence- to-sequence neural machine translation (NMT) model towards the well-studied statistical word alignment models. We show that our novel guided alignment training approach improves trans- lation quality on real-life e-commerce texts consisting of product titles and descriptions, over- coming the problems posed by many unknown words and a large type/token ratio. We also show that meta-data associated with input texts such as topic or category information can sig- nificantly improve translation quality when used as an additional signal to the decoder part of the network. With both novel features, the BLEU score of the NMT system on a product title set improves from 18.6 to 21.3%. Even larger MT quality gains are obtained through domain adaptation of a general domain NMT system to e-commerce data. The developed NMT system also performs well on the IWSLT speech translation task, where an ensemble of four variant systems outperforms the phrase-based baseline by 2.1% BLEU absolute. 1 Introduction NMT systems were shown to reach state-of-the-art translation quality on tasks established in MT research community such as IWSLT speech translation task Cettolo et al. (2012). In this paper, we also apply NMT approach to e-commerce data: user-generated product titles and descriptions for items put on sale. Such data are very different from newswire and other texts typically considered in the MT research community. Titles in particular are short (usually fewer than 15 words), contain many brand names which often do not have to be translated, but also product feature values and specific abbreviations and jargon. Also, the vocabulary size is very large due to the large variety of product types, and many words are observed in the training data only once. At the same time, these data are provided with additional meta-information about the item (e.g. product category such as clothing or electronics), which can be used as context to perform topic/domain adaptation for improved translation quality. At first glance, established phrase-based statistical MT approaches are well-suited for e- commerce data translation. In a phrase-based approach, singleton, but unambiguous words and phrases are usually translated correctly. Also, since the alignment between source and ������������������������������������������������������ ����������������������������������������� target words is available, it is possible to transfer certain entities from the source sentence to the generated target sentence “in-context” without translating them. Such entities can include numbers, product specifications such as “5S” or brand names such as “Samsung” or “Lenovo”. In training, these entities can be replaced with placeholders to reduce the vocabulary size. However, NMT approaches are more powerful at capturing context beyond phrase bound- aries and were shown to better exploit available training data. They also successfully adapt themselves to a domain, for …

Content truncated to 500 words