Selecting Informative Context Sentence by Forced Back-Translation

Selecting Informative Context Sentence by Forced Back-Translation Ryuichiro Kimura†, Shohei Iida†, Hongyi Cui†, Po-Hsuan Hung†, Takehito Utsuro† and Masaaki Nagata‡ †Graduate School of Systems and Information Engineering, University of Tsukuba, Japan ‡NTT Communication Science Laboratories, NTT Corporation, Japan Abstract As one of the contributions of this paper, this paper first explores the upper bound of context-based neural machine transla- tion and attempt to utilize previously un- used context information. We found that, if we could appropriately select the most in- formative context sentence for a given in- put source sentence, we could boost trans- lation accuracy as much as approximately 10 BLEU points. This paper next explores a criterion to select the most informa- tive context sentences that give the high- est BLEU score. Applying the proposed criterion, context sentences that yield the highest forced back-translation probability when back-translating into the source sen- tence are selected. Experimental results with Japanese and English parallel sen- tences from the OpenSubtitles2018 corpus demonstrate that, when the context length of five preceding and five subsequent sen- tences are examined, the proposed ap- proach achieved significant improvements of 0.74 (Japanese to English) and 1.14 (En- glish to Japanese) BLEU scores compared to the baseline 2-to-2 model, where the or- acle translation achieved upper bounds im- provements of 5.88 (Japanese to English) and 9.10 (English to Japanese) BLEU scores. 1 Introduction Recently, neural machine translation (NMT) mod- els (Sutskever et al., 2014; Luong et al., 2015; c⃝ 2019 The authors. This article is licensed under a Creative Commons 4.0 licence, no derivative works, attribution, CC- BY-ND. Vaswani et al., 2017) have made remarkable progress. Most NMT models are designed to translate a single sentence and do not accept in- put greater than one sentence, i.e., input sentences that include additional context information. How- ever, recently, several approaches that attempt to translate inputs with more than one sentence have been proposed (Tiedemann and Scherrer, 2017; Li- bovick´y and Helcl, 2017; Maruf and Haffari, 2018; Miculicich et al., 2018; Bawden et al., 2018; Voita et al., 2018; Tu et al., 2018). These approaches to context-based NMT models can be roughly cate- gorized according to the width of the context con- sidered in those models. A typical approach is to consider the sentence immediately preceding the source sentence to be translated as the con- text (Tiedemann and Scherrer, 2017; Libovick´y and Helcl, 2017; Bawden et al., 2018; Voita et al., 2018). Context-based NMT models can be fur- ther categorized according to whether the source and context sentences are encoded using a sin- gle (Tiedemann and Scherrer, 2017) or multiple encoders (Libovick´y and Helcl, 2017; Bawden et al., 2018; Voita et al., 2018). Another approach considers a much wider context than the imme- diately preceding sentence, e.g., three preceding sentences (Miculicich et al., 2018), preceding sen- tences within the document (Tu et al., 2018), and all preceding and subsequent sentences within the document (Maruf and Haffari, 2018). Such approaches to context-based NMT models possibly outperform existing models that only ac- …

Content truncated to 500 words