Fast-Paced Improvements to Named Entity Handling for Neural Machine Translation

Fast-Paced Improvements to Named Entity Handling for Neural Machine Translation Pedro Mota, Vera Cabarrão, Eduardo Farah Unbabel, Lisbon, Portugal {pedro.mota, vera.cabarrao, eduardo.farah}@unbabel.com Abstract In this work, we propose a Named Entity (NE) handling approach to improve trans- lation quality within an existing Natural Language Processing (NLP) pipeline with- out modifying the Neural Machine Trans- lation (NMT) component. Our approach seeks to enable fast delivery of such im- provements and alleviate user experience problems related to NE distortion. We implement separate NE recognition and translation steps. Then, a combination of standard entity masking technique and a novel semantic equivalent placeholder guarantees that both NE translation is re- spected and the best overall quality is obtained from NMT. The experiments show that translation quality improves in 38.6% of the test cases when compared to a version of the NLP pipeline with less- developed NE handling capability. 1 Introduction NE play a crucial role in many downstream NLP tasks. There is extensive research showing that properly handling NE improves the performance of systems performing Question Answering (Tal- mor and Berant, 2019), Summarization (Zhou et al., 2021), and Information Retrieval (Wang et al., 2021). In this paper, we focus on NMT, another task that benefits from NE modeling (Shavarani and Sarkar, 2021). NMT models are prone to dis- turb NE, leading to critical quality issues in the translation. Overcoming such problems is chal- lenging since it is hard to have good coverage © 2022 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- BY-ND. of all possible entities in the training data. This is due to the open-ended nature of NE as well as their domain specificity. For example, for the Organization (ORG) category, new entities appear daily in a variety of domains. Moreover, NE are linguistically complex structures that can occur in ambiguous contexts. This impairs the ability of models to generalize and instead learn unwanted biases (Hassan Awadalla et al., 2018; Modrzejewski et al., 2020). This causes NE to be hallucinated towards frequent realizations, omit- ted, or incorrectly translated. Figure 1 shows some examples of this issue in the output translation of an English → French NMT model. This occurs despite the model having 65 × 106 parameters and being trained with 100 million sentence pairs. The NMT community has long been famil- iar with the NE handling problem (Koehn and Knowles, 2017). This has spurred research on how to address such model limitations. Invariably, all works resort to either incorporating new model- ing features in existing NMT architectures (Li et al., 2019; Modrzejewski et al., 2020) or integrat- ing with external knowledge sources to bridge the NE gap (Zhao et al., 2020a; Feng et al., 2021). In spite of the achievements of the previously mentioned works, they have the drawback of re- quiring a model-specific solution. In a commercial setting, this is problematic since NE handling, at least for some categories, might come only as an afterthought. Having the NLP pipeline already in …

Content truncated to 500 words