Measuring and Mitigating Name Biases in Neural Machine Translation

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 2576 - 2590 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Measuring and Mitigating Name Biases in Neural Machine Translation Jun Wang and Benjamin I. P. Rubinstein and Trevor Cohn University of Melbourne, Australia jun2@student.unimelb.edu.au {benjamin.rubinstein,trevor.cohn}@unimelb.edu.au Abstract Neural Machine Translation (NMT) systems exhibit problematic biases, such as stereotypi- cal gender bias in the translation of occupation terms into languages with grammatical gender. In this paper we describe a new source of bias prevalent in NMT systems, relating to transla- tions of sentences containing person names. To correctly translate such sentences, a NMT sys- tem needs to estimate the gender of names. We show that leading systems are particularly poor at this task, especially for female given names. This bias is deeper than given name gender: we show that the translation of terms with am- biguous sentiment can also be affected by per- son names, and the same holds true for proper nouns denoting race. To mitigate these biases we propose a simple but effective data augmen- tation method based on randomly switching entities during translation, which effectively eliminates the problem without any effect on translation quality. 1 Introduction Natural language processing systems are seeing widespread adoption, prompting careful study into cultural biases they exhibit, and methods for bias mitigation. Gender bias is common in automated systems (Park et al., 2018; Borkan et al., 2019; Stanovsky et al., 2019; Saunders and Byrne, 2020), with a leading cause being training corpora that include far more sentences referring to men than to women. A neural machine translation (NMT) system naïvely trained on such data is more likely to translate text that should be feminine into mas- culine when translating into a language with gram- matical gender. Previously, researchers (Stanovsky et al., 2019; Escudé Font and Costa-jussà, 2019; Saunders and Byrne, 2020; Stafanovics et al., 2020) have demonstrated that NMT systems can still be biased even when there are explicit gender pro- nouns in the input sentences. NMT systems are not only biased for gender, and gender bias is not limited to gender pronouns. Other biases include racial biases, professional bi- ases, and individual biases, among others. In this paper, we focus on two kinds of biases of person name translations by NMT systems: gender biases and sentiment biases. As an important category of named entity, person names are particularly sen- sitive to translation errors since they refer to real- world individuals, and systematic biases may cause serious distress to users, and reputational damage, libel or other legal consequences for vendors. Gender bias in the translation of person names is a natural extension of gender biases in previ- ous work. For instance, (Stanovsky et al., 2019; Escudé Font and Costa-jussà, 2019; Saunders and Byrne, 2020; Stafanovics et al., 2020) considered whether translation systems can translate keywords such as occupation terms into the correct form when there is explicit gender information in the text. This paper can be …

Content truncated to 500 words