A Case Study on the Importance of Named Entities in a Machine Translation Pipeline for Customer Support Content

A Case Study on the Importance of Named Entities in a Machine Translation Pipeline for Customer Support Content Miguel Menezes1,2 Vera Cabarrão3 Pedro Mota3 Helena Moniz1,2 Alon Lavie3 1 Universidade de Lisboa, Lisboa, Portugal 2 INESC-ID, Lisboa, Portugal 3 Unbabel, Lisboa, Portugal {lmenezes, helena.moniz}@campus.ul.pt {vera.cabarrao, pedro.mota, alon.lavie}@unbabel.com Abstract This paper describes research developed at Unbabel, a Portugal-based translation technology company, that combines MT with human post-edition and focuses mainly on customer service content. We aim to contribute to furthering translation quality and good-practices by exposing the importance of having a continuously- in-development robust Named Entity Recognition system that, among other advantages, supports General Data Protection Regulation (GDPR) compliance. Moreover, we have tested semi-automatic strategies that support and enhance the creation of Named Entities gold standards to allow a more seamless implementation of Multilingual Named Entities Recognition Systems. The project described in this paper is the result of a shared work between Unbabel´s linguists and Unbabel´s AI engineering team, matured over a year. The project should also be taken as a statement of multidisciplinarity, proving and validating the much-needed articulation between the different scientific fields that compose and characterize the area of Natural Language Processing (NLP). © 2022 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- BY-ND. 1 Introduction Customer support professionals deal with multiple issues and problems arising from human- interaction, from answering questions or responding to customer complaints, to processing orders and returns, as well as sharing information and services. They are, in a sense, a direct line between customers and service providers, so they must be efficient, fast, and overall understandable, all while working remotely. Unbabel enhances customer support abilities through the combination of a Machine Translation (MT) layer, coupled with human post-edition, allowing to combine the speed and scale of MT with the quality of human editing. To that end, we focus on Named Entity Recognition processes that compose a vital part of the automatic translation pipeline, since they promote an increase in translation quality, and ensure 2018 data protection regulation compliance. To promote high MT performances, a Named Entity Recognition System (NER) was applied, enabling the identification of NEs in context, e.g., prediction of NEs according to its surroundings, while simultaneously categorizing the NE. The identified NEs are then automatically blocked for translation or automatically annotated as NE of interest for further processes such as localization. This step ensures a decrease in MT “hallucinations” (inadequate translations) (Lee et al., 2018), since NEs are often responsible for these severe MT mistranslations, which negatively impacts the overall translation quality, considered mostly as critical errors in terms of severity. There is a second step associated with the NE pipeline, the anonymization process. The anonymization guarantees that all the NEs corresponding to personal identifiable information (PII) are either replaced by an adequate placeholder, for example Email; Phone Number; Reference Number; or replaced by a semantic equivalent (Mota et al., 2022) in case the NE is a person's name. In the latter, …

Content truncated to 500 words