Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7724–7736 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 7724 Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem Danielle Saunders and Bill Byrne Department of Engineering, University of Cambridge, UK {ds636, wjb31}@cam.ac.uk Abstract Training data for NLP tasks often exhibits gen- der bias in that fewer sentences refer to women than to men. In Neural Machine Translation (NMT) gender bias has been shown to reduce translation quality, particularly when the target language has grammatical gender. The recent WinoMT challenge set allows us to measure this effect directly (Stanovsky et al., 2019). Ideally we would reduce system bias by sim- ply debiasing all data prior to training, but achieving this effectively is itself a challenge. Rather than attempt to create a ‘balanced’ dataset, we use transfer learning on a small set of trusted, gender-balanced examples. This approach gives strong and consistent improve- ments in gender debiasing with much less com- putational cost than training from scratch. A known pitfall of transfer learning on new do- mains is ‘catastrophic forgetting’, which we address both in adaptation and in inference. During adaptation we show that Elastic Weight Consolidation allows a performance trade-off between general translation quality and bias re- duction. During inference we propose a lattice- rescoring scheme which outperforms all sys- tems evaluated in Stanovsky et al. (2019) on WinoMT with no degradation of general test set BLEU, and we show this scheme can be applied to remove gender bias in the output of ‘black box‘ online commercial MT systems. We demonstrate our approach translating from English into three languages with varied lin- guistic properties and data availability. 1 Introduction As language processing tools become more preva- lent concern has grown over their susceptibility to social biases and their potential to propagate bias (Hovy and Spruit, 2016; Sun et al., 2019). Natu- ral language training data inevitably reflects biases present in our society. For example, gender bias manifests itself in training data which features more examples of men than of women. Tools trained on such data will then exhibit or even amplify the bi- ases (Zhao et al., 2017). Gender bias is a particularly important problem for Neural Machine Translation (NMT) into gender- inflected languages. An over-prevalence of some gendered forms in the training data leads to trans- lations with identifiable errors (Stanovsky et al., 2019). Translations are better for sentences involv- ing men and for sentences containing stereotypical gender roles. For example, mentions of male doc- tors are more reliably translated than those of male nurses (Sun et al., 2019; Prates et al., 2019). Recent approaches to the bias problem in NLP have involved training from scratch on artificially gender-balanced versions of the original dataset (Zhao et al., 2018; Zmigrod et al., 2019) or with de- biased embeddings (Escud´e Font and Costa-juss`a, 2019; Bolukbasi et al., 2016). While these ap- proaches may be effective, training from scratch is inefficient and gender-balancing embeddings …

Content truncated to 500 words