GhostBERT: Generate More Features with Cheap Operations for BERT

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 600–616 August 1–6, 2021. ©2021 Association for Computational Linguistics 600 Understanding and Countering Stereotypes: A Computational Approach to the Stereotype Content Model Kathleen C. Fraser, Isar Nejadgholi, and Svetlana Kiritchenko National Research Council Canada Ottawa, Canada {Kathleen.Fraser,Isar.Nejadgholi,Svetlana.Kiritchenko}@nrc-cnrc.gc.ca Abstract Stereotypical language expresses widely-held beliefs about different social categories. Many stereotypes are overtly negative, while others may appear positive on the surface, but still lead to negative consequences. In this work, we present a computational approach to inter- preting stereotypes in text through the Stereo- type Content Model (SCM), a comprehensive causal theory from social psychology. The SCM proposes that stereotypes can be under- stood along two primary dimensions: warmth and competence. We present a method for defining warmth and competence axes in se- mantic embedding space, and show that the four quadrants defined by this subspace accu- rately represent the warmth and competence concepts, according to annotated lexicons. We then apply our computational SCM model to textual stereotype data and show that it com- pares favourably with survey-based studies in the psychological literature. Furthermore, we explore various strategies to counter stereo- typical beliefs with anti-stereotypes. It is known that countering stereotypes with anti- stereotypical examples is one of the most ef- fective ways to reduce biased thinking, yet the problem of generating anti-stereotypes has not been previously studied. Thus, a better under- standing of how to generate realistic and effec- tive anti-stereotypes can contribute to address- ing pressing societal concerns of stereotyping, prejudice, and discrimination. 1 Introduction Stereotypes are widely-held beliefs about traits or characteristics of groups of people. While we tend to think of stereotypes as expressing negative views of groups, some stereotypes actually express posi- tive views (e.g. all women are nurturing). However, even so-called ‘positive’ stereotypes can be harm- ful, as they dictate particular roles that individuals are expected to fulfill, regardless of whether they have the ability or desire to do so (Kay et al., 2013). The existence of stereotypes in our society – in- cluding in entertainment, the workplace, public dis- course, and even legal policy – can lead to a number of harms. Timmer (2011) organizes these harms into three main categories: (1) Misrecognition ef- fects: harms caused by denying members of partic- ular groups an equal place in society, diminishing their human dignity, or other forms of marginaliza- tion. (2) Distribution effects: harms resulting from unfair allocation of resources, either by increas- ing the burden placed on a group, or decreasing a group’s access to a benefit. (3) Psychological effects: the distress and unhappiness caused by an awareness and internalization of the stereotyped biases against one’s identity group. Additionally, the internalization of these negative stereotypes can lead to anxiety and underachievement. To reduce these harms and promote a more egalitarian so- ciety, we must identify and counter stereotypical language when it occurs. Evidence from the psychological literature sug- gests that one …

Content truncated to 500 words