Domain Classification-based Source-specific Term Penalization for Domain Adaptation in Hate-speech Detection

Proceedings of the 29th International Conference on Computational Linguistics, pages 6656–6666 October 12–17, 2022. 6656 Domain Classification-based Source-specific Term Penalization for Domain Adaptation in Hate-speech Detection Tulika Bose † Nikolaos Aletras ‡ Irina Illina † Dominique Fohr † † Universite de Lorraine, CNRS, Inria, LORIA, F-54000 Nancy, France ‡University of Sheffield, United Kingdom {tulika.bose, illina, dominique.fohr}@loria.fr n.aletras@sheffield.ac.uk Abstract Warning: this paper contains content that may be offensive and distressing. State-of-the-art approaches for hate-speech de- tection usually exhibit poor performance in out- of-domain settings. This occurs, typically, due to classifiers overemphasizing source-specific information that negatively impacts its domain invariance. Prior work has attempted to penal- ize terms related to hate-speech from manu- ally curated lists using feature attribution meth- ods, which quantify the importance assigned to input terms by the classifier when making a prediction. We, instead, propose a domain adaptation approach that automatically extracts and penalizes source-specific terms using a do- main classifier, which learns to differentiate be- tween domains, and feature-attribution scores for hate-speech classes, yielding consistent im- provements in cross-domain evaluation. 1 Introduction While recent state-of-the-art hate-speech classi- fiers (Ayo et al., 2021; D’Sa et al., 2020; Mozafari et al., 2019) yield impressive performance on in- domain held-out instances, they suffer when evalu- ated on out-of-domain settings (Yin and Zubiaga, 2021; Arango et al., 2019; Swamy et al., 2019; Karan and Šnajder, 2018). The distributions across corpora/domains1 change due to varying vocabu- lary, topics of discussion over time (Florio et al., 2020; Saha and Sindhwani, 2012), data bias caused by sampling strategies (Wiegand et al., 2019) and different hate-targets. This is concerning since cu- rating new data resources for hate-speech involves substantial time and effort (Poletto et al., 2019; Mal- masi and Zampieri, 2018). This calls for strategies, like Domain Adaptation (DA) approaches, that can adapt models trained on existing labeled resources to a new target domain that lacks class-labels. 1We use the terms ‘corpus’ and ‘domain’ interchangeably. However, research on DA in hate-speech is lim- ited (Sarwar and Murdock, 2022; Bashar et al., 2021; Bose et al., 2021). Typically, vanilla clas- sifiers tend to learn more from domain-specific features (Ye et al., 2021; Wiegand et al., 2019) than domain-invariant features, resulting in poor out-of- domain performance. For instance, Wiegand et al. (2019) show that in a hate-speech dataset (Waseem and Hovy, 2016), neutral domain-specific terms, like ‘football’, ‘commentator’, etc., discussing the role of women in sports, are highly correlated with the hate label, restricting its generalizability. Thus, it is worth minimizing the importance of such terms for improving cross-domain performance. Recently, feature attributions – methods for ex- tracting post-hoc model explanations, have been used to align features with prior domain knowledge (Rieger et al., 2020; Adebayo et al., 2020). These provide importance scores to the input terms as per their contribution towards the model prediction (Lundberg and Lee, 2017). For instance, Liu and Avci (2019); Kennedy et al. (2020) reduce the over- sensitivity of classifiers on a curated list of identity terms (e.g. muslims, gay) by penalizing …

Content truncated to 500 words