Contextualizing Hate Speech Classifiers with Post-hoc Explanation

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5435–5442 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 5435 Contextualizing Hate Speech Classifiers with Post-hoc Explanation Brendan Kennedy∗and Xisen Jin∗and Aida Mostafazadeh Davani Morteza Dehghani and Xiang Ren University of Southern California {btkenned,xisenjin,mostafaz,mdehghan,xiangren}@usc.edu Abstract Hate speech classifiers trained on imbalanced datasets struggle to determine if group identi- fiers like “gay” or “black” are used in offen- sive or prejudiced ways. Such biases mani- fest in false positives when these identifiers are present, due to models’ inability to learn the contexts which constitute a hateful usage of identifiers. We extract post-hoc explana- tions from fine-tuned BERT classifiers to de- tect bias towards identity terms. Then, we propose a novel regularization technique based on these explanations that encourages models to learn from the context of group identifiers in addition to the identifiers themselves. Our approach improved over baselines in limiting false positives on out-of-domain data while maintaining or improving in-domain perfor- mance.† 1 Introduction Hate speech detection is part of the ongoing effort to limit the harm done by oppressive and abusive language (Waldron, 2012; Gelber and McNamara, 2016; Gagliardone et al., 2015; Mohan et al., 2017). Performance has improved with access to more data and more sophisticated algorithms (e.g., Mon- dal et al., 2017; Silva et al., 2016; Del Vigna12 et al., 2017; Basile et al., 2019), but the relative sparsity of hate speech requires sampling using keywords (e.g., Olteanu et al., 2018) or sampling from environments with unusually high rates of hate speech (e.g., de Gibert et al., 2018; Hoover et al., 2019). Modern text classifiers thus struggle to learn a model of hate speech that generalizes to real-world applications (Wiegand et al., 2019). A specific problem found in neural hate speech classifiers is their over-sensitivity to group iden- tifiers like “Muslim”, “gay”, and “black”, which are only hate speech when combined with the right ∗ Authors contributed equally † Code is available here “[F]or many Africans, the most threatening kind of ethnic hatred is black against black.” - New York Times “There is a great discrepancy between whites and blacks in SA. It is … [because] blacks will always be the most backward race in the world.” Anonymous user, Gab.com Figure 1: Two documents which are classified as hate speech by a fine-tuned BERT classifier. Group identi- fiers are underlined. context (Dixon et al., 2018). In Figure 1 we see two documents containing the word “black” that a fine- tuned BERT model predicted to be hate speech, while only the second occurs in a hateful context. Neural text classifiers achieve state-of-the-art performance in hate speech detection, but are un- interpretable and can break when presented with unexpected inputs (Niven and Kao, 2019). It is thus difficult to contextualize a model’s treatment of identifier words. Our approach to this problem is to use the Sampling and Occlusion (SOC) explanation algorithm, which estimates model-agnostic, post- hoc feature importance (Jin et al., 2020). We apply this …

Content truncated to 500 words