Benchmarking Intersectional Biases in NLP

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3598 - 3609 July 10-15, 2022 ©2022 Association for Computational Linguistics Benchmarking Intersectional Biases in NLP John P. Lalor,1,2 Yi Yang,3, Kendall Smith,1,2 Nicole Forsgren,4 Ahmed Abbasi1,2 1 Human-centered Analytics Lab, University of Notre Dame 2 Department of IT, Analytics, and Operations, University of Notre Dame 3 Department of Information Systems and Operations Management, HKUST 4 Microsoft Research {john.lalor, ksmith77, aabbasi}@nd.edu, imyiyang@ust.hk, nicole.forsgren@microsoft.com Abstract There has been a recent wave of work assess- ing the fairness of machine learning models in general, and more specifically, on natural language processing (NLP) models built us- ing machine learning techniques. While much work has highlighted biases embedded in state- of-the-art language models, and more recent efforts have focused on how to debias, research assessing the fairness and performance of bi- ased/debiased models on downstream predic- tion tasks has been limited. Moreover, most prior work has emphasized bias along a sin- gle dimension such as gender or race. In this work, we benchmark multiple NLP models with regards to their fairness and predictive performance across a variety of NLP tasks. In particular, we assess intersectional bias - fair- ness across multiple demographic dimensions. The results show that while current debiasing strategies fare well in terms of the fairness- accuracy trade-off (generally preserving predic- tive power in debiased models), they are unable to effectively alleviate bias in downstream tasks. Furthermore, this bias is often amplified across demographic dimensions. We conclude with implications for future NLP debiasing research. 1 Introduction As state-of-the-art natural language processing (NLP) language models become increasingly pow- erful and pervasive, recent progress in NLP has underscored the need for deeper analyses of how such models perform with respect to underrepre- sented groups. Research on fairness in NLP has shown that distributed representations of words often encode stereotypes - particularly towards dif- ferent demographic groups (Blodgett et al., 2020; Bender et al., 2021). There is a growing stream of research that looks at mitigating these biases, espe- cially when it manifests in the learned embedding state (Bolukbasi et al., 2016; Zmigrod et al., 2019; Kaneko and Bollegala, 2021). While prior work has undoubtedly moved the needle, recent surveys and research articles have identified several impor- tant gaps and issues (Blodgett et al., 2020; Tan and Celis, 2019). First, much of the current work on ex- amining NLP bias (and proposing debiasing strate- gies) has focused on representational harm - how a model describes certain groups, including stereo- typing and other misrepresentations (Blodgett et al., 2020; Suresh and Guttag, 2019). Conversely, there has been far less work exploring allocational harm in downstream NLP prediction tasks - when a sys- tem distributes resources or opportunities differ- ently (Blodgett et al., 2020; Suresh and Guttag, 2019). Downstream tasks, such as sequence clas- sification, also affect underrepresented groups, as these models show disparate impact on various demographic subsets, including women, African Americans, and the elderly (Blodgett et …

Content truncated to 500 words