Deep One-Class Hate Speech Detection Model

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 7040–7048 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 7040 Deep One-Class Hate Speech Detection Model Saugata Bose, Guoxin Su University of Wollongong Northfields Ave, Wollongong, Australia sb632@uowmail.edu.au, guoxin@uow.edu.au Abstract Hate speech detection for social media posts is considered as a binary classification problem in existing approaches, largely neglecting distinct attributes of hate speeches from other sentimental types such as “aggressive” and “racist”. As these sentimental types constitute a significant major portion of data, the classification performance is compromised. Moreover, those classifiers often do not generalize well across different datasets due to a relatively small number of hate-class samples. In this paper, we adopt a one-class perspective for hate speech detection, where the detection classifier is trained with hate-class samples only. Our model employs a BERT-BiLSTM module for feature extraction and a one-class SVM for classification. A comprehensive evaluation with four benchmarking datasets demonstrates the better performance of our model than existing approaches, as well as the advantage of training our model with a combination of the four datasets. Keywords: BERT, BiLSTM, One-Class SVM, outlier detection, transfer learning, hate class. 1. Introduction Hate speech detection in social media posts differs from information retrieval from traditional documents due to length constraint. Moreover, the challenge mounts up due to insufficient publicly available hate speech datasets (MacAvaney et al., 2019), the absence of benchmark datasets (Swamy et al., 2019), and the fuzzy boundary between hate speech and cyberbully- ing, abusive language, discrimination, profanity, tox- icity, flaming, extremism and radicalization concepts (Fortuna and Nunes, 2018; Poletto et al., 2020). Early works modelled the hate speech detection as a bi- nary classification problem. Since non-hate posts con- tain a variety of sentiment types which do not share the same characteristics, forcing those samples into one class, as opposed to the hate class, leads to low per- formance. In view of this, we consider this problem as a one-class problem, where we determine which in- stances stand out as being dissimilar to the hate class. Those instances are treated as outliers and can be iden- tified effectively by a one-class model such as One- class Support Vector Machine (OC-SVM) (Sch¨olkopf, 2001). ... Deep Features ... ... Classification Score ... Figure 1: Deep One-Class Hate Speech Detection Model. In this paper, we propose a novel hate speech detec- tion model (see Figure 1) which includes a deep learn- ing network for feature extraction and the one-class SVM for classification. The deep learning network that we use constitutes the Transformer-based encoder architecture BERT and a Bi-directional Long Short- Term Memory (BiLSTM) model. We evaluate our model with four publicly available datasets: David- son (Davidson et al., 2017), SemEval-2019 (Task-5, Subtask-A) (Basile et al., 2019), HASOC-2019 (Sub- task B) (Mandl et al., 2019) and Stormfront (de Gibert et al., 2018).1 Moreover, we combine the four datasets to experiment with the generalizability of our model. We show that our model achieves better …

Content truncated to 500 words