GFST: Gender-Filtered Self-Training for More Accurate Gender in Translation

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1640–1654 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 1640 GFST: Gender-Filtered Self-Training for More Accurate Gender in Translation Prafulla Kumar Choubey∗† Texas A&M University prafulla.choubey@tamu.edu Anna Currey∗, Prashant Mathur, Georgiana Dinu Amazon AI Translate {ancurrey,pramathu,gddinu}@amazon.com Abstract Targeted evaluations have found that machine translation systems often output incorrect gen- der in translations, even when the gender is clear from context. Furthermore, these incor- rectly gendered translations have the potential to reflect or amplify social biases. We propose gender-filtered self-training (GFST) to improve gender translation accuracy on unambiguously gendered inputs. Our GFST approach uses a source monolingual corpus and an initial model to generate gender-specific pseudo-parallel cor- pora which are then filtered and added to the training data. We evaluate GFST on translation from English into five languages, finding that it improves gender accuracy without damaging generic quality. We also show the viability of GFST on several experimental settings, includ- ing re-training from scratch, fine-tuning, con- trolling the gender balance of the data, forward translation, and back-translation.1 1 Introduction Recent work has drawn attention to the harms that machine learning algorithms can cause by reflect- ing or even amplifying data biases against protected groups (Barocas et al., 2019; Kearns and Roth, 2019). For the most part, machine translation (MT) studies on bias have focused on gender bias in neu- ral machine translation (NMT) and have identified a series of representational harms and stereotyp- ing.2 For example, on input sentences that are un- derspecified in terms of gender, MT models often ∗Equal contribution. †Work done as an intern at Amazon AI Translate. 1Code and data are available at https://github. com/amazon-research/gfst-nmt. 2Following the taxonomy of Blodgett et al. (2020), repre- sentational harms occur when a model’s performance is lower on input data associated with a protected group as opposed to other groups. Stereotyping occurs when a model’s prediction reflects negative stereotypes, for example about a specific eth- nicity, or other stereotypical correlations, for example between professions and gender. Monolingual English Masculine Src Feminine Src Baseline Model Masculine Trg Feminine Trg Masculine Out Feminine Out Parallel Data Source Filtering Target Morphological Filtering Combined Parallel Data Final Model Figure 1: GFST approach for NMT. default to masculine or gender-stereotypical out- puts (Cho et al., 2019; Prates et al., 2018), which can exclude female and non-binary people (e.g., the sentence I am a doctor spoken by a woman may be translated incorrectly as I am a (male) doctor). Even on unambiguously gendered inputs, NMT models can exhibit poorer performance, in terms of overall quality or gender translation accuracy, on content with non-masculine referents (Bentivogli et al., 2020; Stanovsky et al., 2019). In this paper, we take on the task of improving gender translation accuracy, focusing on unambigu- ous inputs where there is only one correct transla- tion with respect to gender. This task is especially difficult when translating from languages with very limited grammatical gender (such as English) into languages with extensive gender …

Content truncated to 500 words