UVA Wahoos at SemEval-2019 Task 6: Hate Speech Identification using Ensemble Machine Learning

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 806–811 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 806 UVA Wahoos at SemEval-2019 Task 6: Hate Speech Identification using Ensemble Machine Learning Murugesan Ramakrishnan Data Science Institue University of Virginia mr6rx@virginia.edu Wlodek Zadrozny Computer Science UNC-Charlotte wzadrozn@uncc.edu Narges Tabari Data Science Institute University of Virginia ns5kn@virginia.edu Abstract With the growth in the usage of social media, it has become increasingly common for peo- ple to hide behind a mask and abuse others. We have attempted to detect such tweets and comments that are malicious in intent, which either targets an individual or a group. Our best classifier for identifying offensive tweets for SubTask A (Classifying offensive vs. non- offensive) has an accuracy of 83.14% and a f1- score of 0.7565 on the actual test data. For SubTask B, to identify if an offensive tweet is targeted (If targeted towards an individual or a group), the classifier performs with an accu- racy of 89.17% and f1-score of 0.5885. The paper talks about how we generated linguis- tic and semantic features to build an ensem- ble machine learning model. By training with more extracts from different sources (Face- book, and more tweets), the paper shows how the accuracy changes with additional training data. 1 Introduction Internet is now accessed by over half of the world’s population 1. In fact, almost 1 million new users are added each day. With social media platforms, people find it a lot easier to get away with the abuse they spew around, in comparison to the offline world. This brings a lot of onus on the Social Network websites to tackle such ac- tivities. Majority of the countries have laws to control hate speech which puts tremendous pres- sure on the concerned websites to curb such activ- ities. Since manual monitoring or defining a spe- cific rule-set might be time consuming, an ensem- ble machine learning approach has been discussed to avoid complexity and increase interpretability. The paper focuses on providing solutions to SubTask A and SubTask B for the SemEval 2019 1https://news.itu.int/ itu-statistics-leaving-no-one-offline/ competition. Previous works and papers focus on identifying if a tweet is offensive or not. Here, in addition to that, it is identified if an offensive tweet is targeted towards a particular individual or a group (SubTask B). Such granular information would help the Social Media to make better deci- sions. 2 Related Work This issue has gathered a lot of attention over the past few years with various types of hate speech detection models. Papers published in the last two years include the surveys by Schmidt and Wiegand (2017) and Fortuna and Nunes (2018) where the authors extract features from the text like sentiment, linguistic features, utilize different lexical re- sources to tag an offensive tweet, and another paper by Davidson et al. (2017) presenting the Hate Speech Detection data set used in Malmasi and Zampieri (2017) where the authors perform a three way classification - Hate …

Content truncated to 500 words