Noisy Self-Training with Data Augmentations for Offensive and Hate Speech Detection Tasks

Proceedings of Recent Advances in Natural Language Processing, pages 631–640 Varna, Sep 4–6, 2023 https://doi.org/10.26615/978-954-452-092-2_068 631 Noisy Self-Training with Data Augmentations for Offensive and Hate Speech Detection Tasks Jo˜ao A. Leite1 and Carolina Scarton1 and Diego F. Silva2 1Department of Computer Science, The University of Sheffield, Sheffield (UK) 2Instituto de Ciˆencias Matem´aticas e de Computac¸˜ao, Universidade de S˜ao Paulo, S˜ao Carlos (Brazil) {jaleite1, c.scarton}@sheffield.ac.uk, diegofsilva@usp.br Abstract Online social media is rife with offensive and hateful comments, prompting the need for their automatic detection given the sheer amount of posts created every second. Creating high- quality human-labelled datasets for this task is difficult and costly, especially because non- offensive posts are significantly more frequent than offensive ones. However, unlabelled data is abundant, easier, and cheaper to ob- tain. In this scenario, self-training methods, using weakly-labelled examples to increase the amount of training data, can be employed. Re- cent “noisy” self-training approaches incorpo- rate data augmentation techniques to ensure prediction consistency and increase robustness against noisy data and adversarial attacks. In this paper, we experiment with default and noisy self-training using three different textual data augmentation techniques across five dif- ferent pre-trained BERT architectures varying in size. We evaluate our experiments on two offensive/hate-speech datasets and demonstrate that (i) self-training consistently improves per- formance regardless of model size, resulting in up to +1.5% F1-macro on both datasets, and (ii) noisy self-training with textual data aug- mentations, despite being successfully applied in similar settings, decreases performance on offensive and hate-speech domains when com- pared to the default method, even with state-of- the-art augmentations such as backtranslation. 1 Introduction Online social media platforms are widely used by modern society for many productive purposes. However, they are also known for intensifying of- fensive and hateful comments, attributed in part to factors such as user anonymity (Mondal et al., 2017). Manual identification of hate speech is impractical at scale due to the massive number of posts generated every second and the potential harm to the mental health of moderators. There- fore, there is a need for automatic approaches to detect offensive and hateful speech. In recent years, research on this topic has in- creased, resulting in new models and datasets pub- lished in various languages and sources (Fortuna and Nunes, 2018). A common characteristic among available datasets is label skewness towards the negative class (non-offensive/hateful), which is usually more frequent than the positive class (of- fensive/hateful). Apart from traditional ways of dealing with imbalanced classes (e.g. under or oversampling or applying class weighting), semi- supervised techniques such as self-training can be used to extend the training set with unseen exam- ples that introduce new learning signals without the costly burden of manual data labeling. Self-training is a technique that involves itera- tively training models using both labelled and unla- belled data. The process begins by training a model using human-labelled data only, which is then used to infer labels for a set of unlabelled data, creat- ing a weakly-labelled dataset. The weakly-labelled dataset and the human-labelled …

Content truncated to 500 words