AdelaideCyC at SemEval-2020 Task 12: Ensemble of Classifiers for Offensive Language Detection in Social Media

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1516–1523 Barcelona, Spain (Online), December 12, 2020. 1516 AdelaideCyC at SemEval-2020 Task 12: Ensemble of Classifiers for Offensive Language Detection in Social Media Mahen Herath1, Thushari Atapattu2, Hoang Anh Dung2, Christoph Treude2, Katrina Falkner2 1Department of Computer Science & Engineering, University of Moratuwa, Sri Lanka 2School of Computer Science, The University of Adelaide, Adelaide, SA 5005, Australia mahenherath@gmail.com, dunganh.hoang@student.adelaide.edu.au, {thushari.atapattu, christoph.treude, katrina.falkner}@adelaide.edu.au Abstract This paper describes the systems our team (AdelaideCyC) has developed for SemEval Task 12 (OffensEval 2020) to detect offensive language in social media. The challenge focuses on three subtasks – offensive language identification (subtask A), offense type identification (subtask B), and offense target identification (subtask C). Our team has participated in all the three subtasks. We have developed machine learning and deep learning-based ensembles of models. We have achieved F1-scores of 0.906, 0.552, and 0.623 in subtask A, B, and C respectively. While our performance scores are promising for subtask A, the results demonstrate that subtask B and C still remain challenging to classify. 1 Introduction The surge of Internet and social media technologies provides a wealth of opportunities for cybercrime, and has led to the unprecedented social crisis of online abuse. Despite the illegality of such behaviour, most social media platforms such as Facebook, Twitter, Instagram are susceptible to online bullying due to their openness and anonymisation. The sheer amount of offensive language generation vastly exceeds the capacity of manual detection. Therefore, there is a crucial need for urgent development of technological solutions. The automated identification of offensive language has been recognised as a subtask of NLP only recently and most of the advances have occurred in the last few years. The offensive language identification as an NLP problem is inherently complex and challenging – even for humans (aside from the offensive language's victims) due to many variants of language used by harassers such as coarse language, sarcasm, intimidation, and colloquialisms. People also tend to use coarse language in a friendly manner, without an intention to harm anyone. Therefore, it is important to identify whether a post or a tweet is offensive and whether it is targeted at an individual or a group. In this paper, we focus on identifying offensive posts extracted from the Twitter platform. The training dataset contained more than 9 million tweets and they were annotated using a semi- supervised approach. Our team has participated in English versions of all three subtasks organised by Zampieri et al. (2020), i.e., subtask A: offensive language identification (offensive or not), subtask B: offense type identification (targeted insults and threats or untargeted), and subtask C: offense target identification (individual, group, other). In this paper, we discuss the models developed for each subtask along with the performance. The advancement of offensive language identification has many benefits for social media and online communities to protect their users. 2 Related work The identification of unacceptable language in social media and online communities has attracted attention from researchers in related …

Content truncated to 500 words