SemEval-2019 Task 5: Multilingual Detection of Hate Speech Against Immigrants and Women in Twitter

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 54–63 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 54 SemEval-2019 Task 5: Multilingual Detection of Hate Speech Against Immigrants and Women in Twitter Valerio Basile♦ Cristina Bosco♦ Elisabetta Fersini♥ Debora Nozza♥ Viviana Patti♦ Francisco Rangel♠♣ Paolo Rosso♣ Manuela Sanguinetti♦ ♦ Dipartimento di Informatica, Universit`a degli Studi di Torino (Italy) ♣ Universit`a degli Studi di Milano Bicocca (Italy) ♠ Autoritas Consulting (Spain) ♥ PRHLT Research Center, Universitat Polit`ecnica de Val`encia (Spain) ♦{name.surname}@unito.it, ♥{name.surname}@unimib.it, ♠francisco.rangel@autoritas.es, ♣prosso@dsic.upv.es Abstract The paper describes the organization of the SemEval 2019 Task 5 about the detection of hate speech against immigrants and women in Spanish and English messages extracted from Twitter. The task is organized in two related classification subtasks: a main binary subtask for detecting the presence of hate speech, and a finer-grained one devoted to identifying fur- ther features in hateful contents such as the ag- gressive attitude and the target harassed, to dis- tinguish if the incitement is against an individ- ual rather than a group. HatEval has been one of the most popular tasks in SemEval-2019 with a total of 108 submitted runs for Subtask A and 70 runs for Subtask B, from a total of 74 different teams. Data provided for the task are described by showing how they have been col- lected and annotated. Moreover, the paper pro- vides an analysis and discussion about the par- ticipant systems and the results they achieved in both subtasks. 1 Introduction Hate Speech (HS) is commonly defined as any communication that disparages a person or a group on the basis of some characteristic such as race, color, ethnicity, gender, sexual orientation, nation- ality, religion, or other characteristics (Nockleby, 2000). Given the huge amount of user-generated contents on the Web, and in particular on social media, the problem of detecting, and therefore possibly contrasting the HS diffusion, is becom- ing fundamental, for instance for fighting against misogyny and xenophobia. Some key aspects feature online HS, such as vi- rality, or presumed anonymity, which distinguish it from offline communication and make it po- tentially also more dangerous and hurtful. Often hate speech fosters discrimination against partic- ular categories and undermines equality, an ever- lasting issue for each civil society. Among the mainly targeted categories there are immigrants and women. For the first target, especially raised by refugee crisis and political changes occurred in the last few years, several governments and pol- icy makers are currently trying to address it, mak- ing especially interesting the development of tools for the identification and monitoring such kind of hate (Bosco et al., 2017). For the second one in- stead, hate against the female gender is a long-time and well-known form of discrimination (Manne, 2017). Both these forms of hate content impact on the development of society and may be con- fronted by developing tools that automatically de- tect them. A large number of academic events and shared tasks for different languages (i.e. English, Span- …

Content truncated to 500 words