Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP

Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP Timo Schick∗ Sahana Udupa† Hinrich Sch¨utze∗ ∗Center for Information and Language Processing (CIS), LMU Munich, Germany †Institute of Social and Cultural Anthropology, LMU Munich, Germany schickt@cis.lmu.de, sahana.udupa@lmu.de, inquiries@cislmu.org Abstract � This paper contains prompts and model outputs that are offensive in nature. When trained on large, unfiltered crawls from the Internet, language models pick up and re- produce all kinds of undesirable biases that can be found in the data: They often generate racist, sexist, violent, or otherwise toxic language. As large models require millions of training exam- ples to achieve good performance, it is difficult to completely prevent them from being ex- posed to such content. In this paper, we first demonstrate a surprising finding: Pretrained language models recognize, to a considerable degree, their undesirable biases and the toxi- city of the content they produce. We refer to this capability as self-diagnosis. Based on this finding, we then propose a decoding algorithm that, given only a textual description of the undesired behavior, reduces the probability of a language model producing problematic text. We refer to this approach as self-debiasing. Self-debiasing does not rely on manually cu- rated word lists, nor does it require any training data or changes to the model’s parameters. While we by no means eliminate the issue of language models generating biased text, we believe our approach to be an important step in this direction.1 1 Introduction Pretraining neural networks using a language mod- eling objective leads to large improvements across a variety of natural language processing tasks (Peters et al., 2018; Radford et al., 2018; Devlin et al., 2019). With model sizes continually in- creasing (Radford et al., 2019; Raffel et al., 2020; Brown et al., 2020; Fedus et al., 2021), ever-larger pretraining datasets are necessary both to pre- vent overfitting and to provide access to as much 1Our implementation is publicly available at https:// github.com/timoschick/self-debiasing. world knowledge as possible. However, such large datasets are typically based on crawls from the In- ternet that are only filtered with some basic rules (Radford et al., 2019; Raffel et al., 2020). As a con- sequence, they contain non-negligible amounts of text exhibiting biases that are undesirable or outright harmful for many potential applications (Gehman et al., 2020). Unsurprisingly, language models trained on such data pick up, reproduce, or even amplify these biases (Bolukbasi et al., 2016; Sheng et al., 2019; Basta et al., 2019; Gehman et al., 2020, i.a.). Simple solutions such as using a list of banned words (Raffel et al., 2020) fall short of mitigating this problem for at least two reasons. First, they do not reliably keep language models from gener- ating biased text: Examples in Figure 1 show that biased text can easily be generated by using only words that are, by themselves, completely un- problematic. As many such words are important words of the English vocabulary and thus needed for meaningful text generation, they should not be included in a list …

Content truncated to 500 words