Modeling Consistency Preference via Lexical Chains for Document-level Neural Machine Translation

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6312 - 6326 December 7-11, 2022 ©2022 Association for Computational Linguistics Modeling Consistency Preference via Lexical Chains for Document-level Neural Machine Translation Xinglin Lyu♣, Junhui Li♣∗, Shimin Tao♠, Hao Yang♠, Ying Qin♠ , Min Zhang♣ ♣School of Computer Science and Technology, Soochow University, Suzhou, China ♠Huawei Translation Services Center, Beijing, China xllv2020@stu.suda.edu.cn {lijunhui,minzhang}@suda.edu.cn {taoshimin,yanghao30,qinying}@huawei.com Abstract In this paper we aim to relieve the issue of lexical translation inconsistency for document- level neural machine translation (NMT) by modeling consistency preference for lexical chains which consist of repeated words in a source-side document and provide a represen- tation of the lexical consistency structure of the document. Specifically, we first propose lexical-consistency attention to capture consis- tency context among words in the same lex- ical chains. Then for each lexical chain we define and learn a consistency-tailored latent variable, which will guide the translation of corresponding sentences to enhance lexical translation consistency. Experimental results on Chinese→English and French→English document-level translation tasks show that our approach not only significantly improves trans- lation performance in BLEU, but also substan- tially alleviates the problem of the lexical trans- lation inconsistency. 1 Introduction Generally, the translations of source-side words repeated within a document tend to be consistent (Merkel, 1996; Carpuat, 2009; Türe et al., 2012; Guillou, 2013; Al Khotaba and Al Tarawneh, 2015; Lyu et al., 2021; Kang et al., 2021) while sentence- level neural machine translation (NMT) suffers from the serious problem of lexical translation in- consistency due to the lack of inter-sentence con- text. Although most recent studies in document- level NMT propose various context-aware models to better capture inter-sentence context, they do not handle specific discourse phenomena, e.g., lex- ical consistency. In this paper, we therefore study lexical consistency for document-level NMT by modeling consistency preference for lexical chains which represent the lexical consistency structure of a text. ∗Corresponding author: Junhui Li. Lexical translation consistency is a common discourse phenomenon. Many studies in statis- tical machine translation (SMT) (Merkel, 1996; Carpuat, 2009; Türe et al., 2012; Guillou, 2013; Al Khotaba and Al Tarawneh, 2015) discuss and apply the one translation per discourse hypothe- sis. More recently, Lyu et al. (2021) give a de- tailed analysis about the lexical translation con- sistency on Chinese→English translation task. In their analysis, the proportion of words related with this phenomenon reaches about 20 percent against all of words in the whole corpus. Furthermore, they find out that the translations of the words repeated within a document indeed tend to be consistent. Intuitively, enhancing lexical translation consis- tency should consider inter-sentence context. How- ever, existing researches in document-level NMT mainly focus on capturing inter-sentence context in general and do not explicitly handle specific discourse phenomena. As a result, these models have limited effect on enhancing lexical transla- tion consistency. Different from them, both Lyu et al. (2021) and Kang et al. (2021) recently in- troduce auxiliary consistency losses to encourage the translations of the repeated source-side …

Content truncated to 500 words