Necessity and Sufficiency for Explaining Text Classifiers: A Case Study in Hate Speech Detection

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2672 - 2686 July 10-15, 2022 ©2022 Association for Computational Linguistics Necessity and Sufficiency for Explaining Text Classifiers: A Case Study in Hate Speech Detection Esma Balkır, Isar Nejadgholi, Kathleen C. Fraser, and Svetlana Kiritchenko National Research Council Canada Ottawa, Canada {Esma.Balkir,Isar.Nejadgholi,Kathleen.Fraser,Svetlana.Kiritchenko}@nrc-cnrc.gc.ca Abstract We present a novel feature attribution method for explaining text classifiers, and analyze it in the context of hate speech detection. Although feature attribution models usually provide a sin- gle importance score for each token, we instead provide two complementary and theoretically- grounded scores – necessity and sufficiency – resulting in more informative explanations. We propose a transparent method that calculates these values by generating explicit perturba- tions of the input text, allowing the importance scores themselves to be explainable. We em- ploy our method to explain the predictions of different hate speech detection models on the same set of curated examples from a test suite, and show that different values of necessity and sufficiency for identity terms correspond to dif- ferent kinds of false positive errors, exposing sources of classifier bias against marginalized groups. 1 Introduction Explainability in AI (XAI) is critical in reaching various objectives during a system’s development and deployment, including debugging the system, ensuring its fairness, safety and security, and under- standing and appealing its decisions by end-users (Vaughan and Wallach, 2021; Luo et al., 2021). A popular class of local explanation techniques is feature attribution methods, where the aim is to provide scores for each feature according to how important that feature is for the classifier decision for a given input. From an intuitive perspective, one issue with feature attribution scores is that it is not always clear how to interpret the assigned im- portance in operational terms. Specifically, saying that a feature is ‘important’ might translate to two different predictions. The first interpretation is that if an important feature value is changed, then the prediction will change. The second interpretation is that, as long as the feature remains, the prediction will not change. The former interpretation corre- sponds to the necessity of the feature value, while the latter corresponds to its sufficiency. To further illustrate the difference between neces- sity and sufficiency, we take an example from hate speech detection. Consider the utterance “I hate women”. For a perfect model, the token ‘women’ should have low sufficiency for a positive predic- tion, since merely the mention of this identity group should not trigger a hateful prediction. However, this token should have fairly high necessity, since a target identity is required for an abusive utter- ance to count as hate speech (e.g., “I hate oranges” should not be classified as hate speech). In this paper, we develop a method to estimate the neces- sity and sufficiency of each word in the input, as explanations for a binary text classifier’s decisions. Model-agnostic feature attribution methods like ours often perturb the input to be …

Content truncated to 500 words