Effective Use of Target-side Context for Neural Machine Translation

Proceedings of the 28th International Conference on Computational Linguistics, pages 4483–4494 Barcelona, Spain (Online), December 8-13, 2020 4483 Effective Use of Target-side Context for Neural Machine Translation Hideya Mino 1,2 Hitoshi Ito 1 Isao Goto 1 Ichiro Yamada 1 Takenobu Tokunaga 2 1 NHK Science & Technology Research Laboratories 1-10-11 Kinuta, Setagaya-ku, Tokyo 157-8510, Japan {mino.h-gq,itou.h-ce,goto.i-es,yamada.i-hy}@nhk.or.jp 2 Tokyo Institute of Technology 2-12-1 Ookayama, Meguro-ku, Tokyo 152-8550, Japan take@c.titech.ac.jp Abstract Through the progress made in a sentence-level neural machine translation (NMT), a context- aware NMT has been rapidly developed to exploit previous sentences as context. Recent work in the context-aware NMT incorporates source- or target-side contexts. In contrast to the source- side context, the target-side context causes a gap between training that utilizes a ground truth sentence and inference using a machine-translated sentence as context. This gap leads to transla- tion quality deteriorating because the translation model is trained with only the ground truth data that cannot be used in the inference. In this paper, we propose sampling both the ground truth and the machine-translated previous sentences of the target-side for the context-aware NMT. The proposed method can make the translation model robust against mistakes and biases made at the inference. Models using our proposed approach show improvements over models using the previous approaches in English ↔ Japanese and English ↔ German translation tasks. 1 Introduction Neural machine translation (NMT) (Sutskever et al., 2014; Bahdanau et al., 2015; Vaswani et al., 2017) has made great progress in translating sentences in isolation. In contrast to isolated sentences, sentences in documents cannot be correctly translated without context outside the current sentence (L¨aubli et al., 2018). To address this problem, various context-aware NMT models have been developed to exploit preceding and/or succeeding sentences in source- and/or target-side languages as context (Wang et al., 2017; Tiedemann and Scherrer, 2017; Bawden et al., 2018; Voita et al., 2018; Voita et al., 2019; Maruf and Haffari, 2018; Maruf et al., 2019; Agrawal et al., 2018; Kuang et al., 2018; Miculicich et al., 2018). Bawden et al. (2018) used NMT models with source- and target-side contexts in the previous sentence, and reported that the source-side context was effective for improving the translation quality. However, the target-side context led to lower quality even though they highlighted the importance of the target-side context. Agrawal et al. (2018) also pointed out that the use of the target-side context increases the risk of error propagation. We considered that the reason for the low quality was a gap between a training and an inference when using target-side context in the model. At the training phase, the ground truth sentences (references) were used as the target-side context, and at the inference phase, the sentences predicted by a translation model were used. The predicted sentence includes translation errors such as mistranslation, under-translation, and over-translation not included in the ground truth sentence. Even if there are no translation errors in the predicted sentence, there is a bias of translationese that is outputs of machine translation tend …

Content truncated to 500 words