Know-Center at SemEval-2019 Task 5: Multilingual Hate Speech Detection on Twitter using CNNs

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 431–435 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 431 Know-Center at SemEval-2019 Task 5: Multilingual Hate Speech Detection on Twitter using CNNs Kevin Winter Know-Center GmbH Inffeldgasse 13 Graz, 8010, Austria kwinter@know-center.at Roman Kern Know-Center GmbH Inffeldgasse 13 Graz, 8010, Austria rkern@know-center.at Abstract This paper presents the Know-Center system submitted for task 5 of the SemEval-2019 workshop. Given a Twitter message in either English or Spanish, the task is to first detect whether it contains hateful speech and second, to determine the target and level of aggres- sion used. For this purpose our system uti- lizes word embeddings and a neural network architecture, consisting of both dilated and tra- ditional convolution layers. We achieved aver- age F1-scores of 0.57 and 0.74 for English and Spanish respectively. 1 Introduction The ever-increasing number of message board fo- rums, social media platforms and other websites that allow user comments enable participants to express their opinions freely and sometimes even anonymously. This barley restricted access in combination with the unmanageably vast amount of user-generated content unfortunately also cre- ates an environment, which is vulnerable to pro- fanity and hateful speech, rendering it hostile to the individuals or groups of people targeted. This problem is of increasing importance (Kettrey and Laster, 2014) and calls to establish systems that allow for automated detection of such behavior. While detecting abusive language by it self is al- ready a challenging task, (Malmasi and Zampieri, 2018) showed that it is even harder to differentiate between its subtypes. Messages containing pro- fanity, sexism, racism and other forms of hateful speech may be formed using very similar vocabu- laries. Additionally, these subtypes may overlap, making data sets dependent on the subjective judg- ments of annotators. This may be particularly true for finding a threshold distinguishing aggressive and none-aggressive speech. In a task organized as part of COLING 2018 (Kumar et al., 2018) 15,000 participants were asked to detect aggressive be- haviour in a data set of Facebook posts. Even the best system only obtained a weighted F1-score of 0.64. Task 5 of the SemEval-2019 workshop (Basile et al., 2019) aims to accelerate the research and de- velopment of such systems. It comprises two dis- tinct subtasks. The first subtask is devoted to de- tect hateful speech against immigrants and women in Twitter messages. For the second subtask, mes- sages that are detected to be hateful are investi- gated further. Here, the goal is to identify ag- gressive behaviour and the target, women or immi- grant, harassed. Given the multilingual nature of the task, systems for both English and Spanish are required. For the design and evaluation of these systems, training data sets for both languages are provided. The English data set consists of 9000 annotated messages whereas the Spanish data set consists of 4500. Closely related, another task is hosted at the SemEval-2019 workshop, dealing with offensive language in social media and the individuals and groups …

Content truncated to 500 words