Studying Generalisability across Abusive Language Detection Datasets

Proceedings of the 23rd Conference on Computational Natural Language Learning, pages 940–950 Hong Kong, China, November 3-4, 2019. c⃝2019 Association for Computational Linguistics 940 Studying Generalisability Across Abusive Language Detection Datasets Steve Durairaj Swamy and Anupam Jamatia Department of Computer Science National Institute of Technology Agartala, India {steve050798,anupamjamatia}@gmail.com Bj¨orn Gamb¨ack∗ Department of Computer Science Norwegian University of Science and Technology Trondheim, Norway gamback@ntnu.no Abstract Work on Abusive Language Detection has tackled a wide range of subtasks and do- mains. As a result of this, there exists a great deal of redundancy and non-generalisability between datasets. Through experiments on cross-dataset training and testing, the paper re- veals that the preconceived notion of including more non-abusive samples in a dataset (to em- ulate reality) may have a detrimental effect on the generalisability of a model trained on that data. Hence a hierarchical annotation model is utilised here to reveal redundancies in exist- ing datasets and to help reduce redundancy in future efforts. 1 Introduction With the growth of the internet and the increas- ingly smaller barrier to entry, social media have become viable platforms for people to make their views known. These easily accessible fora for discourse have given a voice to many minori- ties and individuals to share their stories. The caveat, however, is that these platforms can be misused to spread hate and harass other individ- uals, which has given birth to terms such as cyber- bullying and trolling. Online harassment has been a point of criticism levied against social media gi- ants such as Facebook and Twitter, who have come under increased pressure to address this misuse. To this end, they have ensured that their commu- nity guidelines explicitly ban the usage of profan- ity/hate speech to harass and bully individuals. The detection of Online Abuse has proven to be a layered and complex issue. For example, pro- fanity is often treated as a sign of hate speech or offensive language, but profanity can also be used in a wide variety of expressive ways to convey in- formality, humour, and emphasis. This usage of ∗Also at: RISE SICS, Kista, Sweden. profanity outside of abuse/insults, coupled with implicit insults that may not contain any profan- ity, makes the task of classifying abuse online a balancing act of sorts, forming the crux of what makes this task hard to tackle: stricter guidelines may hamper a well-meaning individual’s freedom of speech, while more lenient guidelines may em- power those who exploit them. As it stands, the intricacies of free speech do not translate well to machine understanding. This has led to the continued use of human modera- tors in the abusive language detection space. Con- tent is flagged by users, reviewed by a human and removed if it violates the platform’s community guidelines. The main problem with this system is the sheer volume of content to be reviewed, giv- ing human moderators very little time to arrive at a decision. Another issue that was highlighted by Roberts (2019) is the impact that …

Content truncated to 500 words