The binary trio at SemEval-2019 Task 5: Multitarget Hate Speech Detection in Tweets

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 489–493 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 489 The binary trio at SemEval-2019 Task 5: Multitarget Hate Speech Detection in Tweets Patricia Chiril IRIT Toulouse University patricia.chiril@irit.fr Farah Benamara IRIT, CNRS Toulouse University benamara@irit.fr V´eronique Moriceau LIMSI-CNRS Univ. Paris-Sud moriceau@limsi.fr Abhishek Kumar Indian Institute of Science abhishekkumar12@iisc.ac.in Abstract The massive growth of user-generated web content through blogs, online forums and most notably, social media networks, led to a large spreading of hatred or abusive messages which have to be moderated. This paper proposes a supervised approach to hate speech detec- tion towards immigrants and women in En- glish tweets. Several models have been de- veloped ranging from feature-engineering ap- proaches to neural ones. We also carried out a detailed error analysis to show main causes of misclassification. 1 Motivation Social media networks such as Facebook, Twit- ter, blogs and forums, have become a space where users are free to relate events, personal expe- riences, but also opinions and sentiments about products, events or other people. This massive growth of user generated web content, along with the interactivity and anonymity the internet pro- vides, may lead to a large spreading of hatred or abusive messages which have to be moderated. In spite of no universally accepted definition of hate speech and the way it differs from offensive language, there are some common elements that seem to arise. In particular, these messages may express threats, harassment, intimidation or ”dis- parage a person or a group on the basis of some characteristic such as race, color, ethnicity, gender, sexual orientation, nationality, religion, or other characteristic” (Nockleby, 2000). In this paper, we focus on automatic hate speech detection towards two different targets – immigrants and women and we propose several multi-target hate speech detection systems. The task is performed over a collection of English tweets annotated as conveying hate speech against both immigrants and women, as part of HateE- val@SemEval2019 (Basile et al., 2019). The first challenge involves building a binary classifier able to determine whether a tweet with a given target (women or immigrants) is hateful or not hateful. For this, we propose both features-based models (relying on both language-dependent and language independent features) and a neural model. We also performed a detailed error analysis to iden- tify main causes of misclassification. Our anal- ysis shows that errors come from several factors, which show the complexity of the task: the pres- ence of irony and sarcasm, the lack of context and implicit hate speech. We also identified tweets for which we question the original label when taking into account the class definition. The paper is organized as follows. Section 2 briefly presents the current state of the art. Section 3 describes our data and models, while Section 4 analayses the experiments we carried out on multi- target detection. We conclude by providing some perspectives for future work. 2 Related work Hateful speech can be expressed at different …

Content truncated to 500 words