Leveraging Discourse Rewards for Document-Level Neural Machine Translation

Proceedings of the 28th International Conference on Computational Linguistics, pages 4467–4482 Barcelona, Spain (Online), December 8-13, 2020 4467 Leveraging Discourse Rewards for Document-Level Neural Machine Translation Inigo Jauregi Unanue1,3, Nazanin Esmaili1, Gholamreza Haffari2, Massimo Piccardi1 1University of Technology Sydney, NSW 2007, Australia 2Monash University, VIC 3800, Australia 3RoZetta Technology, NSW 2000, Australia Inigo.Jauregi@rozettatechnology.com {Nazanin.Esmaili,Massimo.Piccardi}@uts.edu.au Gholamreza.Haffari@monash.edu Abstract Document-level machine translation focuses on the translation of entire documents from a source to a target language. It is widely regarded as a challenging task since the translation of the in- dividual sentences in the document needs to retain aspects of the discourse at document level. However, document-level translation models are usually not trained to explicitly ensure discourse quality. Therefore, in this paper we propose a training approach that explicitly optimizes two established discourse metrics, lexical cohesion (LC) and coherence (COH), by using a reinforce- ment learning objective. Experiments over four different language pairs and three translation domains have shown that our training approach has been able to achieve more cohesive and co- herent document translations than other competitive approaches, yet without compromising the faithfulness to the reference translation. In the case of the Zh-En language pair, our method has achieved an improvement of 2.46 percentage points (pp) in LC and 1.17 pp in COH over the runner-up, while at the same time improving 0.63 pp in BLEU score and 0.47 pp in FBERT. 1 Introduction The recent advances in neural machine translation (NMT) (Sutskever et al., 2014; Bahdanau et al., 2015; Luong et al., 2015; Vaswani et al., 2017) have provided the research community and the commercial landscape with effective translation models that can at times achieve near-human performance. However, this usually holds at phrase or sentence level. When using these models in larger units of text, such as paragraphs or documents, the quality of the translation may drop considerably in terms of discourse attributes such as lexical and stylistic consistency. In fact, document-level translation is still a very open and challenging problem. The sentences that make up a document are not unrelated pieces of text that can be predicted independently; rather, a set of sequences linked together by complex underlying linguistics aspects, also known as the discourse (Maruf et al., 2019b; Jurafsky and Martin, 2019). The discourse of a document includes several properties such as grammatical cohesion (Halliday and Hasan, 2014), lexical cohesion (Halliday and Hasan, 2014), docu- ment coherence (Hobbs, 1979) and the use of discourse connectives (Kalajahi et al., 2012). Ensuring that the translation retain such linguistic properties is expected to significantly improve its overall readability and flow. However, due to the limitations of current decoder technology, NMT models are still bound to translate at sentence level. In order to capture the discourse properties of the source document in the translation, re- searchers have attempted to incorporate more contextual information from surrounding sentences. Most document-level NMT approaches augment the model with multiple encoders, extra attention layers and memory caches to encode the surrounding sentences, and leave the model to implicitly …

Content truncated to 500 words