Improving Neural Machine Translation through Phrase-based Forced Decoding

Proceedings of the The 8th International Joint Conference on Natural Language Processing, pages 152–162, Taipei, Taiwan, November 27 – December 1, 2017 c⃝2017 AFNLP Improving Neural Machine Translation through Phrase-based Forced Decoding Jingyi Zhang1,2, Masao Utiyama1, Eiichro Sumita1 Graham Neubig3,2, Satoshi Nakamura2 1National Institute of Information and Communications Technology, Japan 2Graduate School of Information Science, Nara Institute of Science and Technology, Japan 3Language Technologies Institute, Carnegie Mellon University, USA jingyizhang/mutiyama/eiichiro.sumita@nict.go.jp gneubig@cs.cmu.edu, s-nakamura@is.naist.jp Abstract Compared to traditional statistical ma- chine translation (SMT), neural machine translation (NMT) often sacrifices ade- quacy for the sake of fluency. We propose a method to combine the advantages of tra- ditional SMT and NMT by exploiting an existing phrase-based SMT model to com- pute the phrase-based decoding cost for an NMT output and then using this cost to rerank the n-best NMT outputs. The main challenge in implementing this approach is that NMT outputs may not be in the search space of the standard phrase-based decoding algorithm, because the search space of phrase-based SMT is limited by the phrase-based translation rule table. We propose a soft forced decoding algorithm, which can always successfully find a de- coding path for any NMT output. We show that using the forced decoding cost to rerank the NMT outputs can success- fully improve translation quality on four different language pairs. 1 Introduction Neural machine translation (NMT), which uses a single large neural network to model the entire translation process, has recently been shown to outperform traditional statistical machine transla- tion (SMT) such as phrase-based machine transla- tion (PBMT) on several translation tasks (Koehn et al., 2003; Bahdanau et al., 2015; Sennrich et al., 2016a). Compared to traditional SMT, NMT gen- erally produces more fluent translations, but of- ten sacrifices adequacy, such as translating source words into completely unrelated target words, over-translation or under-translation (Koehn and Knowles, 2017). There are a number of methods that combine the two paradigms to address their respective weak- nesses. For example, it is possible to incorpo- rate neural features into traditional SMT models to disambiguate hypotheses (Neubig et al., 2015; Stahlberg et al., 2016). However, the search space of traditional SMT is usually limited by transla- tion rule tables, reducing the ability of these mod- els to generate hypotheses on the same level of fluency as NMT, even after reranking. There are also methods that incorporate knowledge from tra- ditional SMT into NMT, such as lexical translation probabilities (Arthur et al., 2016; He et al., 2016), phrase memory (Tang et al., 2016; Zhang et al., 2017), and n-gram posterior probabilities based on traditional SMT translation lattices (Stahlberg et al., 2017). These improve the adequacy of NMT outputs, but do not impose hard alignment con- straints like traditional SMT systems and there- fore cannot effectively solve all over-translation or under-translation problems. In this paper, we propose a method that ex- ploits an existing phrase-based translation model to compute the phrase-based decoding cost for a given NMT translation.1 That is, we force a phrase-based translation system to …

Content truncated to 500 words