NIT_Agartala_NLP_Team at SemEval-2019 Task 6: An Ensemble Approach to Identifying and Categorizing Offensive Language in Twitter Social Media Corpora

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 696–703 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 696 NIT Agartala NLP Team at SemEval-2019 Task 6: An Ensemble Approach to Identifying and Categorizing Offensive Language in Twitter Social Media Corpora Steve Durairaj Swamy1, Anupam Jamatia1, Bj¨orn Gamb¨ack2 and Amitava Das3 1National Institute of Technology, Agartala, India 2Norwegian University of Science and Technology, Trondheim, Norway 3Mahindra ´Ecole Centrale,Hyderabad, Telangana, India {steve050798,anupamjamatia}@gmail.com,gamback@ntnu.no,amitava.das@mechyd.ac.in Abstract The paper describes the systems submit- ted to OffensEval (SemEval 2019, Task 6) on ‘Identifying and Categorizing Offen- sive Language in Social Media’ by the ‘NIT Agartala NLP Team’. A Twitter anno- tated dataset of 13,240 English tweets was pro- vided by the task organizers to train the indi- vidual models, with the best results obtained using an ensemble model composed of six dif- ferent classifiers. The ensemble model pro- duced macro-averaged F1-scores of 0.7434, 0.7078 and 0.4853 on Subtasks A, B, and C, respectively. The paper highlights the over- all low predictive nature of various linguistic features and surface level count features, as well as the limitations of a traditional machine learning approach when compared to a Deep Learning counterpart. 1 Introduction Offensive language has been the scourge of the in- ternet since the rise of social media. Social me- dia provides a platform for everyone and anyone to voice their opinion. This has empowered peo- ple to make their voices heard and to speak out on global issues. The downside to this, however, is the misuse of such platforms to attack an individ- ual or a minority group, and to spread hateful opin- ions. Pairing this with the perceived anonymity the internet provides, there has been a massive up- swing in the use of social media for cyberbullying and hate speech, with technology giants coming under increased pressure to address the issue. Most of what we may be interested in detect- ing can be broadly labelled as hate speech, cyber- bullying or abusive use of swearing. The union of these three subsets form what can be identified as ‘Offensive Language on Social Media’. However, what we consider offensive is often a grey area, as is evident by the low inter-annotator agreement rates when labelling data for offensive language (Waseem et al., 2017b). Detecting offensive language has proven to be difficult, due to the broad spectrum in which lan- guage can be used to convey an insult. The na- ture of the abuse can be implicit — drawing from sarcasm and humour rather than offensive terms — as well as explicit, by making extensive use of traditional offensive terms and profanity. It does not help that the reverse is also entertained, with profanity often being used to imply informality in speech or for emphasis. Coincidentally, these are also the reasons why lexical detection methods have been unfruitful in classifying text as offensive or non-offensive. The OffensEval 2019 shared task (Zampieri et al., 2019b) is one of several endeavours to fur- ther the …

Content truncated to 500 words