An Evaluation of Source Factors in Concatenation-Based Context-Aware Neural Machine Translation

Proceedings of Recent Advances in Natural Language Processing, pages 399–407 Varna, Sep 4–6, 2023 https://doi.org/10.26615/978-954-452-092-2_045 399 An Evaluation of Source Factors in Concatenation-based Context-aware Neural Machine Translation Harritxu Gete1,2, Thierry Etchegoyhen1 1Vicomtech Foundation, Basque Research and Technology Alliance (BRTA) 2University of the Basque Country UPV/EHU {hgete,tetchegoyhen}@vicomtech.org Abstract We explore the use of source factors in context- aware neural machine translation, specifically concatenation-based models, to improve the translation quality of inter-sentential phenom- ena. Context sentences are typically concate- nated to the sentence to be translated, with string-based markers to separate the latter from the former. Although previous studies have measured the impact of prefixes to identify and mark context information, the use of learnable factors has only been marginally explored. In this study, we evaluate the impact of single and multiple source context factors in English- German and Basque-Spanish contextual trans- lation. We show that this type of factors can significantly enhance translation accuracy for phenomena such as gender and register coher- ence in Basque-Spanish, while also improving BLEU results in some scenarios. These results demonstrate the potential of factor-based con- text identification as a research path in context- aware machine translation. 1 Introduction Machine translation typically operates at the sen- tence level, leaving aside larger context informa- tion. This mode of operation remains dominant within the Neural Machine Translation (NMT) framework (Sutskever et al., 2014; Bahdanau et al., 2015; Vaswani et al., 2017), although it limits ac- curate translation for linguistic phenomena that depend on context information, such as cohesion, discourse coherence or intersentential anaphora res- olution (Bawden et al., 2018; L¨aubli et al., 2018; Voita et al., 2019b; Lopes et al., 2020; Post and Junczys-Dowmunt, 2023). Addressing discourse-related phenomena in translation requires extending the scope of the trans- lation models to address the relevant information present in the context sentences, in addition to that of the sentence to be translated. Several approaches have been proposed within NMT to extend the mod- elling window beyond isolated sentences, extend- ing the input by including context sentences (Tiede- mann and Scherrer, 2017) or modifying the NMT architecture to model context information (Jean et al., 2017; Zhang et al., 2018; Voita et al., 2019b; Li et al., 2020). Despite the marked improvements achievable with the aforementioned approaches, the identifica- tion of the relevant contextual information to im- prove the translation of a given sentence is still an open research topic. Within concatenation-based approaches (Tiedemann and Scherrer, 2017), a sim- ple yet strong document-level NMT baseline, con- text sentences are typically prepended to the sen- tence to be translated, and separated from it by a simple marker. Further identification of what belongs to the context or to the sentence to be trans- lated is typically discarded, following in part initial results by Tiedemann and Scherrer (2017) where the use of prefixes to identify context tokens led to degraded results at best. An alternative method that may provide better context identification is the utilization of factors as context markers. Factors are learnable embeddings associated to …

Content truncated to 500 words