Exploring Data Augmentation Strategies for Hate Speech Detection in Roman Urdu

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 4523–4531 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 4523 Exploring Data Augmentation Strategies for Hate Speech Detection in Roman Urdu Ubaid Azam, Hammad Rizwan, Asim Karim Syed Babar Ali School of Science and Engineering, Lahore University of Management Sciences, Lahore, Pakistan {19030040, hammad.rizwan, akarim}@lums.edu.pk Abstract In an era where social media platform users are growing rapidly, there has been a marked increase in hateful content being generated; to combat this; automatic hate speech detection systems are a necessity. For this purpose, researchers have recently focused their efforts on developing datasets, however, the vast majority of them have been generated for the English language, with only a few available for low-resource languages such as Roman Urdu. Furthermore, what few are available have small number of samples that pertain to hateful classes and these lack variations in topics and content. Thus, deep learning models trained on such datasets perform poorly when deployed in the real world. To improve performance the option of collecting and annotating more data can be very costly and time consuming. Thus, data augmentation techniques need to be explored to exploit already available datasets to improve model generalizability. In this paper, we explore different data augmentation techniques for the improvement of hate speech detection in Roman Urdu. We evaluate these augmentation techniques on two datasets. We are able to improve performance in the primary metric of comparison (F1 and Macro F1) as well as in recall, which is impertinent for human-in-the-loop AI systems. Keywords: Hate Speech Detection, Data Augmentation, Sentiment Classification 1. Introduction In recent years, there has been an exponential increase in the number of users on social media platforms such as Twitter, Facebook, etc. Users love to share their opinions and emotions publicly on these platforms and interact with other users. These platforms provide users with the ability to disseminate information to millions of people worldwide within seconds. Users are granted freedom of speech to post whatever they desire given that it is within respectable bounds; but a lot of users also negatively exploit this freedom. Hateful content on these platforms is on the rise. Users are being tar- geted based on gender, religion, race e.t.c. They are being subjected to humiliation and bullying, which can cause rather adverse side effects such as psychological harm, which can result in increased amounts of anxiety experienced and in the worst case has led users to com- mit suicide. According to (Hinduja and Patchin, 2019) the chance of committing suicide increases two-folds if a person is subjected to cyberbullying. Social media platforms are now actively finding and removing hate- ful content by utilizing reporting mechanisms with con- tent moderators and AI-based models. Although they have been able to clamp down on a significant portion of hateful content but the clampdowns influence can only be largely seen on hateful content generated in the English language and for regions where English is pri- …

Content truncated to 500 words