Towards Neural Machine Translation with Partially Aligned Corpora

Proceedings of the The 8th International Joint Conference on Natural Language Processing, pages 384–393, Taipei, Taiwan, November 27 – December 1, 2017 c⃝2017 AFNLP Towards Neural Machine Translation with Partially Aligned Corpora Yining Wang†‡, Yang Zhao†‡, Jiajun Zhang†‡, Chengqing Zong†‡∗ and Zhengshan Xue# †National Laboratory of Pattern Recognition, CASIA, Beijing, China ‡University of Chinese Academy of Sciences, Beijing, China ∗CAS Center for Excellence in Brain Science and Intelligence Technology, Shanghai, China #Toshiba (China) Co.,Ltd. {yining.wang, yang.zhao, jjzhang, cqzong}@nlpr.ia.ac.cn xuezhengshan2@toshiba.com.cn Abstract While neural machine translation (NMT) has become the new paradigm, the pa- rameter optimization requires large-scale parallel data which is scarce in many do- mains and language pairs. In this paper, we address a new translation scenario in which there only exists monolingual cor- pora and phrase pairs. We propose a new method towards translation with partially aligned sentence pairs which are derived from the phrase pairs and monolingual corpora. To make full use of the partially aligned corpora, we adapt the conventional NMT training method in two aspects. On one hand, different generation strategies are designed for aligned and unaligned tar- get words. On the other hand, a different objective function is designed to model the partially aligned parts. The experiments demonstrate that our method can achieve a relatively good result in such a translation scenario, and tiny bitexts can boost trans- lation quality to a large extent. 1 Introduction Neural machine translation (NMT) proposed by Kalchbrenner et al.(2013), Sutskever et al.(2014) and Cho et al.(2014) has achieved significant progress in recent years. Different from traditional statistical machine translation(SMT) (Koehn et al., 2003; Chiang, 2005; Liu et al., 2006; Zhai et al., 2012) which contains multiple separately tuned components, NMT builds an end-to-end frame- work to model the whole translation process. For several language pairs, NMT is reaching signif- icantly better translation performance than SMT (Luong et al., 2015b; Wu et al., 2016). In general, in order to obtain an NMT model 外交部发言人刘建超今天在例行的机制招待会上说， 美国司法部长来北京进行了访问。 Speaking at a regular press briefing Wednesday, Turkish foreign ministry deputy spokesman said that turkey hoped the peace talks would continue as planned. Partial Aligned Part 2 Partial Aligned Part 1 Figure 1: An example of our partially aligned training data, in which the source sentence and tar- get sentence are not parallel but they include two parallel parts (highlight in blue and red respec- tively). of great translation quality, we usually need large- scale parallel data. Unfortunately, the large-scale parallel data is always insufficient in many do- mains and language pairs. Without sufficient par- allel sentence pairs, NMT tends to learn poor esti- mates on low-count events. Actually, there have been some effective meth- ods to deal with the situation of translating lan- guage pairs with limited resource under different scenarios (Johnson et al., 2016; Cheng et al., 2017; Sennrich et al., 2016a; Zhang and Zong, 2016). In this paper, we address a new translation sce- nario in which we do not have any parallel sen- tences but have massive monolingual corpora and phrase …

Content truncated to 500 words