Grunn2019 at SemEval-2019 Task 5: Shared Task on Multilingual Detection of Hate

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 391–395 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 391 Grunn2019 at SemEval-2019 Task 5: Shared Task on Multilingual Detection of Hate Mike Zhang, Roy David, Leon Graumans, Gerben Timmerman University of Groningen Groningen, the Netherlands {j.j.zhang.1, r.a.david, l.r.n.graumans, g.h.r.timmerman}@student.rug.nl Abstract Hate speech occurs more often than ever and polarizes society. To help counter this polar- ization, SemEval 2019 organizes a shared task called the Multilingual Detection of Hate. The first task (A) is to decide whether a given tweet contains hate against immigrants or women, in a multilingual perspective, for English and Spanish. In the second task (B), the system is also asked to classify the following sub- tasks: hateful tweets as aggressive or not ag- gressive, and to identify the target harassed as individual or generic. We evaluate multiple models, and finally combine them in an en- semble setting. This ensemble setting is built of five and three submodels for the English and Spanish task respectively. In the current setup it shows that using a bigger ensemble for English tweets performs mediocre, while a slightly smaller ensemble does work well for detecting hate speech in Spanish tweets. Our results on the test set for English show 0.378 macro F1 on task A and 0.553 macro F1 on task B. For Spanish the results are significantly higher, 0.701 macro F1 on task A and 0.734 macro F1 for task B. 1 Introduction The increasing popularity of social media plat- forms such as Twitter for both personal and politi- cal communication has seen a well-acknowledged rise in the presence of toxic and abusive speech on these platforms (Kshirsagar et al., 2018). Al- though the terms of services on these platforms typically forbid hateful and harassing speech, the volume of data requires that ways are found to classify online content automatically. The prob- lem of detecting, and therefore possibly limit the hate speech diffusion, is becoming fundamental (Nobata et al., 2016). Previous work concerning hate speech against immigrants and women such as Olteanu et al. (2018) observed that extremist violence tends to lead to an increase in online hate speech, partic- ularly on messages directly advocating violence. Also, Anzovino et al. (2018) contributed to the re- search field by (1) making a corpus of misogynous tweets, labelled from different perspective and (2) created an exploratory investigations on NLP fea- tures and ML models for detecting and classifying misogynistic language. Basile et al. (2019) proposed a shared task on the Multilingual Detection of Hate, where partic- ipants have to detect hate speech against immi- grants and women in Twitter, in a multilingual perspective, for English and Spanish. The task is divided in two related subtasks for both lan- guages: a basic task about hate speech, and an- other one where fine-grained features of hateful contents will be investigated in order to understand how existing approaches may deal with the identi- fication of especially dangerous forms of hate, for …

Content truncated to 500 words