FERMI at SemEval-2019 Task 5: Using Sentence embeddings to Identify Hate Speech Against Immigrants and Women in Twitter

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 70–74 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 70 Fermi at SemEval-2019 Task 5: Using Sentence Embeddings to identify Hate Speech against Immigrants and Women on Twitter Vijayasaradhi Indurthi1,3, Bakhtiyar Syed1, Manish Shrivastava1 Nikhil Chakravartula3, Manish Gupta1,2, Vasudeva Varma1 1 IIIT Hyderabad, India 2 Microsoft, India 3 Teradata, India 1{vijaya.saradhi, syed.b}@research.iiit.ac.in 1{m.shrivastava, manish.gupta, vv}@iiit.ac.in 2gmanish@microsoft.com 3nikhil.chakravartula@teradata.com Abstract This paper describes our system (Fermi) for Task 5 of SemEval-2019: HatEval: Multilin- gual Detection of Hate Speech Against Immi- grants and Women on Twitter. We participated in the subtask A for English and ranked first in the evaluation on the test set. We evaluate the quality of multiple sentence embeddings and explore multiple training models to eval- uate the performance of simple yet effective embedding-ML combination algorithms. Our team - Fermi’s model achieved an accuracy of 65.00% for English language in task A. Our models, which use pretrained Universal En- coder sentence embeddings for transforming the input and SVM (with RBF kernel) for clas- sification, scored first position (among 68) in the leaderboard on the test set for Subtask A in English language. In this paper we provide a detailed description of the approach, as well as the results obtained in the task. 1 Introduction Microblogging platforms like Twitter provide channels to exchange ideas using short messages called tweets. While such a platform can be used for constructive ideas, a small group of people can propagate their notions including hatred against an individual, or a group or a race to the entire world in a few seconds. This necessitates the need to come up with computational methods to identify hate speech in user generated content. Using computational methods to identify of- fense, aggression and hate speech in user gener- ated content has been gaining attention in the re- cent years as evidenced in (Waseem et al., 2017; Davidson et al., 2017; Malmasi and Zampieri, 2017; Kumar et al., 2018) and workshops such as Abusive Language Workshop (ALW) 1 and Work- 1https://sites.google.com/view/alw2018 shop on Trolling, Aggression and Cyberbullying (TRAC) 2. 2 Related Work In this section we briefly describe other work in this area. A few of the early works related to hate speech detection employed the use of features like bag of words, word and character n-grams with relatively off-the-shelf machine learning classifiers for de- tection (Dinakar et al., 2011; Waseem and Hovy, 2016; Nobata et al., 2016). Deep learning methods for hate speech detection were used by Badjatiya et al. (2017) wherein the authors experimented with a combination of multiple deep learning ar- chitectures along with randomly initialized word embeddings learned by Long Short Term Memory (LSTM) models. Papers published in the last two years include the surveys by (Schmidt and Wiegand, 2017) and (Fortuna and Nunes, 2018), the paper by (David- son et al., 2017) which presented the Hate Speech Detection dataset used in (Malmasi and Zampieri, 2017) and a few other recent papers such …

Content truncated to 500 words