Guiding Neural Machine Translation with Retrieved Translation Pieces

Proceedings of NAACL-HLT 2018, pages 1325–1335 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Guiding Neural Machine Translation with Retrieved Translation Pieces Jingyi Zhang1,2, Masao Utiyama1, Eiichro Sumita1 Graham Neubig3,2, Satoshi Nakamura2 1National Institute of Information and Communications Technology, Japan 2Graduate School of Information Science, Nara Institute of Science and Technology, Japan 3Language Technologies Institute, Carnegie Mellon University, USA jingyizhang/mutiyama/eiichiro.sumita@nict.go.jp gneubig@cs.cmu.edu, s-nakamura@is.naist.jp Abstract One of the difficulties of neural machine trans- lation (NMT) is the recall and appropriate translation of low-frequency words or phrases. In this paper, we propose a simple, fast, and effective method for recalling previously seen translation examples and incorporating them into the NMT decoding process. Specifically, for an input sentence, we use a search engine to retrieve sentence pairs whose source sides are similar with the input sentence, and then collect n-grams that are both in the retrieved target sentences and aligned with words that match in the source sentences, which we call “translation pieces”. We compute pseudo- probabilities for each retrieved sentence based on similarities between the input sentence and the retrieved source sentences, and use these to weight the retrieved translation pieces. Fi- nally, an existing NMT model is used to trans- late the input sentence, with an additional bonus given to outputs that contain the col- lected translation pieces. We show our method improves NMT translation results up to 6 BLEU points on three narrow domain trans- lation tasks where repetitiveness of the target sentences is particularly salient. It also causes little increase in the translation time, and com- pares favorably to another alternative retrieval- based method with respect to accuracy, speed, and simplicity of implementation. 1 Introduction Neural machine translation (NMT) (Bahdanau et al., 2014; Sennrich et al., 2016a; Wang et al., 2017b) is now the state-of-the-art in machine translation, due to its ability to be trained end-to- end on large parallel corpora and capture complex parameterized functions that generalize across a variety of syntactic and semantic phenomena. However, it has also been noted that compared to alternatives such as phrase-based translation (Koehn et al., 2003), NMT has trouble with low- frequency words or phrases (Arthur et al., 2016; Kaiser et al., 2017), and also generalizing across domains (Koehn and Knowles, 2017). A num- ber of methods have been proposed to ameliorate these problems, including methods that incorpo- rate symbolic knowledge such as discrete trans- lation lexicons (Arthur et al., 2016; He et al., 2016; Chatterjee et al., 2017) and phrase tables (Zhang et al., 2017; Tang et al., 2016; Dahlmann et al., 2017), adjust model structures to be more conducive to generalization (Nguyen and Chiang, 2017), or incorporate additional information about domain (Wang et al., 2017a) or topic (Zhang et al., 2016) in translation models. In particular, one paradigm of interest is recent work that augments NMT using retrieval-based models, retrieving sentence pairs from the training corpus that are most similar to the sentence that we want to translate, and then using these to bias the …

Content truncated to 500 words