Bad Seeds: Evaluating Lexical Methods for Bias Measurement

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 1889–1904 August 1–6, 2021. ©2021 Association for Computational Linguistics 1889 Bad Seeds: Evaluating Lexical Methods for Bias Measurement Maria Antoniak Cornell University maa343@cornell.edu David Mimno Cornell University mimno@cornell.edu Abstract A common factor in bias measurement meth- ods is the use of hand-curated seed lexicons, but there remains little guidance for their selec- tion. We gather seeds used in prior work, docu- menting their common sources and rationales, and in case studies of three English-language corpora, we enumerate the different types of social biases and linguistic features that, once encoded in the seeds, can affect subsequent bias measurements. Seeds developed in one context are often re-used in other contexts, but documentation and evaluation remain neces- sary precursors to relying on seeds for sensi- tive measurements. 1 Introduction There has been increasing concern in the NLP com- munity over bias and stereotypes contained in mod- els and how these biases can trickle downstream to practical applications, such as serving job ad- vertisements. In particular, there has been much recent scrutiny of word representations, with many studies finding harmful associations encoded in em- bedding models. Combating such biases requires measuring the bias encoded in a model so that re- searchers can establish improvements, and many variants of embedding-based measurement tech- niques have been proposed (Bolukbasi et al., 2016; Caliskan et al., 2017; Manzini et al., 2019). These measurements have had the additional up- stream benefit of providing computational social science and digital humanities scholars with a new means of quantifying bias in datasets of social, po- litical, or literary interest. Researchers increasingly use embeddings (Garg et al., 2018; Knoche et al., 2019a; Hoyle et al., 2019) and other lexicon-based methods (Saez-Trumper et al., 2013; Fast et al., 2016; Rudinger et al., 2017) to provide quantitative answers to otherwise elusive political and social Target Concept Highlighted Seeds Unpleasant divorce, jail, poverty, cancer, ... African American Tanisha, Tia, Lakisha, Latoya, ... Domestic Work mom, mum, ... Ugliness fat, chubby, obese, fatty, overweight, disformed, disfigured, wrinkle, wrinkled, ... Table 1: Examples of real seed terms used in recent work to measure biases in corpora. questions about the biases in a corpus and its au- thors. This work often involves comparing bias measurements across different corpora, which re- quires reliable, fine-grained measurements. While there is a wide range of bias measure- ment methods, every one of them relies on lexi- cons of seed terms to specify stereotypes or dimen- sions of interest. But the rationale for choosing specific seeds is often unclear; sometimes seeds are crowd-sourced, sometimes hand-selected by re- searchers, and sometimes drawn from prior work in the social sciences. The impact of the seeds is not well-understood, and many previous seed sets have serious limitations. As shown in Table 1, the seeds used for bias measurement can themselves exhibit cultural and cognitive biases (e.g., reduc- tive definitions), and in addition, linguistic features …

Content truncated to 500 words