A Smorgasbord of Features to Combine Phrase-Based and Neural Machine Translation

A Smorgasbord of Features to Combine Phrase-Based and Neural Machine Translation Benjamin Marie bmarie@nict.go.jp Atsushi Fujita atsushi.fujita@nict.go.jp National Institute of Information and Communications Technology, 3-5 Hikaridai, Seika-cho, Soraku-gun, Kyoto, 619-0289, Japan Abstract Superiority of neural machine translation (NMT) and phrase-based statistical machine transla- tion (PBSMT) depends on the translation task. For some translation tasks, such as those involv- ing low-resource language pairs or close languages, NMT may underperform PBSMT. In order to have a translation system that performs consistently better regardless of the translation task, recent work proposed to combine PBSMT and NMT approaches. In this paper, we propose an empirical comparison of the most popular existing approaches that combine PBSMT and NMT. Despite its simplicity, our simple reranking system using a smorgasbord of informative features significantly and consistently outperforms other methods, even for translation tasks where PBSMT and NMT produce translations of a very different quality. 1 Introduction Neural machine translation (NMT) systems have been shown to outperform phrase-based statis- tical machine translation (PBSMT) systems in many translation tasks. NMT systems perform especially well with language pairs involving two distant languages or morphologically-rich languages. Translations produced by NMT systems are usually more fluent than those produced by state-of-the-art PBSMT systems. However, NMT systems are still far from producing perfect translations. Many researchers have studied the weaknesses of the NMT approach and shown that NMT systems perform poorly compared to PBSMT systems in relatively common sce- narios, especially those involving low-resource language pairs (Bentivogli et al., 2016; Koehn and Knowles, 2017). Several approaches have recently been proposed to combine PBSMT and NMT in order to exploit their complementarity and to produce better translations. In this paper, we study the most popular combination methods and empirically compare them, aiming at drawing a better picture of their strengths and weaknesses. We demonstrate that reranking the simple concatenation of n-best lists produced by each of the NMT and PBSMT systems, with a set of well-motived features, performs consistently the best compared to the more popular and complex methods proposed by previous work. We also show that, while other approaches can perform worse than the best system, a simple reranking approach offers some guarantee that the selected best translation will be rarely worse than the best one proposed by the PBSMT or by the NMT system, even when one of the systems performs very poorly. The remainder of this paper is organized as follows. In Section 2, we review the existing methods used to combine PBSMT and NMT. Then, in Section 3, we make our assumption that a reranking system using a large set of informative features can outperform other existing methods. We evaluate our proposed reranking systems in Section 4 and analyze our results in Section 5. In Section 6, we conclude and propose promising perspectives for this work. Proceedings of AMTA 2018, vol. 1: MT Research Track Boston, March 17 - 21, 2018 | Page 111 2 Current Approaches to Combine PBSMT and NMT This section reviews four different methods able to …

Content truncated to 500 words