He is very intelligent, she is very beautiful? On Mitigating Social Biases in Language Modelling and Generation

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4534–4545 August 1–6, 2021. ©2021 Association for Computational Linguistics 4534 He is very intelligent, she is very beautiful? On Mitigating Social Biases in Language Modelling and Generation Aparna Garimella1, Akhash Amarnath2∗, Kiran Kumar Rathlavath2∗, Akash Pramod Yalla2∗, Anandhavelu N1, Niyati Chhaya1 and Balaji Vasan Srinivasan1 1Adobe Research 2Indian Institute of Technology Madras {garimell,anandvn,nchhaya,balsrini}@adobe.com, {naakhash24,rathlavathkirankumar2,pammuyap}@gmail.com Abstract Social biases with respect to demographics (e.g., gender, age, race) in datasets are often encoded in the large pre-trained language mod- els trained on them. Prior works have largely focused on mitigating biases in context-free representations, with recent shift to contextual ones. While this is useful for several word and sentence-level classification tasks, mitigating biases in only the representations may not suf- fice to use these models for language gener- ation tasks, such as auto-completion, summa- rization, or dialogue generation. In this paper, we propose an approach to mitigate social bi- ases in BERT, a large pre-trained contextual language model, and show its effectiveness in fill-in-the-blank sentence completion and sum- marization tasks. In addition to mitigating bi- ases in BERT, which in general acts as an en- coder, we propose lexical co-occurrence-based bias penalization in the decoder units in gener- ation frameworks, and show bias mitigation in summarization. Finally, our approach results in better debiasing of BERT-based representa- tions compared to post training bias mitigation, thus illustrating the efficacy of our approach to not just mitigate biases in representations, but also generate text with reduced biases. 1 Introduction Bias can be defined as any kind of preference or prejudice toward a specific individual, group, or community over others (Moss-Racusin et al., 2012; Sun et al., 2019). Unstructured data often con- tain several biases, and natural language processing (NLP) models trained on them learn and sometimes amplify them (Bolukbasi et al., 2016; Kurita et al., 2019; Sheng et al., 2019). In this paper, we focus on a specific type of bias called representation bias, where certain groups are associated with certain ∗This work was done when the authors were at Adobe Research. He is very intelligent. She is very beautiful. The man had a job as manager at the company. The woman had a job as receptionist at the company. My father works as a doctor and my mother as a nurse. The Caucasian man is very handsome. The Black man is very angry. The Caucasian woman was known for beauty. The Black woman was known for violence. Table 1: Example sentence completions using BERT. identities, e.g., man is to computer programmer as woman is to homemaker (Bolukbasi et al., 2016). Biases in large contextual language models such as BERT (Devlin et al., 2019) and GPT (Rad- ford et al., 2019) have been receiving increased attention; Tan and Celis (2019) and Zhao et al. (2019) analyzed the extent to which contextual word representations encode gender and racial bi- ases, Caliskan et al. (2016), Kurita et al. (2019) and May et al. (2019) proposed methods to mea- …

Content truncated to 500 words