Measuring Geographic Performance Disparities of Offensive Language Classifiers

Proceedings of the 29th International Conference on Computational Linguistics, pages 6600–6616 October 12–17, 2022. 6600 Measuring Geographic Performance Disparities of Offensive Language Classifiers Brandon Lwowski1, Paul Rad,1,2 and Anthony Rios1 1Department of Information Systems and Cyber Security 2Department of Computer Science University of Texas at San Antonio {brandon.lwowski, peyman.najafirad, anthony.rios}@utsa Abstract Text classifiers are applied at scale in the form of one-size-fits-all solutions. Nevertheless, many studies show that classifiers are biased re- garding different languages and dialects. When measuring and discovering these biases, some gaps present themselves and should be ad- dressed. First, “Does language, dialect, and top- ical content vary across geographical regions?” and secondly “If there are differences across the regions, do they impact model performance?”. We introduce a novel dataset called GeoOLID with more than 14 thousand examples across 15 geographically and demographically diverse cities to address these questions. We perform a comprehensive analysis of geographical-related content and their impact on performance dis- parities of offensive language detection models. Overall, we find that current models do not gen- eralize across locations. Likewise, we show that while offensive language models produce false positives on African American English, model performance is not correlated with each city’s minority population proportions. Warn- ing: This paper contains offensive language. 1 Introduction Many tasks revolving around text classification of social network data have been introduced in- cluding, but not limited to tracking viruses (Lamb et al., 2013; Corley et al., 2009, 2010; Santil- lana et al., 2015; Ahmed et al., 2018; Lwowski and Najafirad, 2020), providing help for (natu- ral) disasters (Neubig et al., 2011; Castillo, 2016; Reuter and Kaufhold, 2018), detecting misinforma- tion (Oshikawa et al., 2020), and identifying cyber- bullying (Xu et al., 2012). Overall, text classifiers have been shown to be “accurate” across a wide range of applications. As deep learning models and packages have made substantial progress for the field of natural language processing (NLP), NLP models have become more accessible to the general public. Hence, models are being deployed in a pro- duction environment and run at scale at a growing pace. However, recent work has shown that these models are biased and unfair, especially towards minority groups (Blodgett et al., 2016; Davidson et al., 2019). In this paper, we expand on prior work by analyzing how model performance can fluctuate due do geographically-caused differences in language and topical content that exists in the context of offensive language detection. Researchers have shown that topical and stylistic attributes of text are used by speakers on social me- dia to implicitly mark their region-of-origin (Shoe- mark et al., 2017; Hovy and Purschke, 2018; Cheke et al., 2020; Gaman et al., 2020). For instance, Hovy and Purschke (2018) show that doc2vec em- bedding frameworks can be leveraged to detect geolocation-related language differences. Hovy et al. (2020) then introduces visualization tech- niques for measuring regional language change. Kellert and Matlis (2021) shows that differences exist at the city level as well. Hence, prior work has generally focused on incorporating or identify- ing …

Content truncated to 500 words