AbuseAnalyzer: Abuse Detection, Severity and Target Prediction for Gab Posts

Proceedings of the 28th International Conference on Computational Linguistics, pages 6277–6283 Barcelona, Spain (Online), December 8-13, 2020 6277 AbuseAnalyzer: Abuse Detection, Severity and Target Prediction for Gab Posts Mohit Chandra1, Ashwin Pathak2, Eesha Dutta3∗, Paryul Jain4∗, Manish Gupta5 Manish Shrivastava6, Ponnurangam Kumaraguru7 International Institute of Information Technology, Hyderabad 1,2,3,4,5,6 Indraprastha Institute of Information Technology, Delhi7 Microsoft5 {mohit.chandra, eesha.dutta, paryul.jain}@research.iiit.ac.in ashwin.pathak@alumni.iiit.ac.in, manish.gupta@iiit.ac.in m.shrivastava@iiit.ac.in, pk@iiitd.ac.in Abstract While extensive popularity of online social media platforms has made information dissemination faster, it has also resulted in widespread online abuse of different types like hate speech, offensive language, sexist and racist opinions, etc. Detection and curtailment of such abusive content is critical for avoiding its psychological impact on victim communities, and thereby preventing hate crimes. Previous works have focused on classifying user posts into various forms of abusive behavior. But there has hardly been any focus on estimating the severity of abuse and the target. In this paper, we present a first of the kind dataset with 7,601 posts from Gab1 which looks at online abuse from the perspective of presence of abuse, severity and target of abusive behavior. We also propose a system to address these tasks, obtaining an accuracy of ∼80% for abuse presence, ∼82% for abuse target prediction, and ∼65% for abuse severity prediction. 1 Introduction In recent times, Online Social Media (OSM) has become an indispensable part of our lives. Not only these websites connect billions of people around the world, but they also serve as a platform for express- ing opinions and sharing information quickly. However, recently OSM platforms have been a subject for criticism over the propagation of fake (Shu et al., 2017) and hateful content (Fortuna and Nunes, 2018). Such cases of online abuse have also translated into real world hate crimes.2 Abuse in social media is spread across a wide spectrum from mild expressions of attitudes and beliefs to strong violent threats. Inspired by hate theories from Anti-Defamation League (ADL)3, we broadly classify forms of abuse as ‘Biased Attitude, ‘Act of Bias and Discrimination’ and ‘Violence and Geno- cide’. Moreover, abusive content could be targeted at specific individuals (e.g., a politician, a celebrity, etc.) or particular groups (a country, LGBTQ+, a religion, gender, an organization, etc.). Detection of such abusive content is critical for avoiding its psychological impact on victim communities, and thereby preventing hate crimes. Prioritization of particular abuse cases can be done if severity of abuse can be automatically assessed. Further, identifying if the abuse target is a person or a large group is critical to predict potential impact set and thereby predict if it could lead to real world crimes along with its scale. Hence, in this paper, we propose three abuse prediction tasks: prediction of abuse presence, abuse severity prediction and abuse target prediction. Since traditional OSM websites are reasonably moderated, finding broadly abusive content is possible. But finding abusive behaviour of differing severity is a ‘needle in a haystack’ kind of challenge. In contrast to the other OSM, Gab is relatively unexplored …

Content truncated to 500 words