STUFIIT at SemEval-2019 Task 5: Multilingual Hate Speech Detection on Twitter with MUSE and ELMo Embeddings

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 464–468 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 464 STUFIIT at SemEval-2019 Task 5: Multilingual Hate Speech Detection on Twitter with MUSE and ELMo Embeddings Michal Bojkovsk´y Faculty of Informatics and Information Technologies STU in Bratislava Ilkoviˇcova 2, Bratislava m.bojkovsky@protonmail.com Mat´uˇs Pikuliak Faculty of Informatics and Information Technologies STU in Bratislava Ilkoviˇcova 2, Bratislava matus.pikuliak@stuba.sk Abstract We evaluate the viability of multilingual learn- ing for the task of hate speech detection. We also experiment with adversarial learning as a means of creating a multilingual model. Ultimately our multilingual models have had worse results than their monolignual counter- parts. We find that the choice of word repre- sentations (word embeddings) is very crucial for deep learning as a simple switch between MUSE and ELMo embeddings has shown a 3- 4% increase in accuracy. This also shows the importance of context when dealing with on- line content. 1 Introduction The Internet has been surging in popularity as well as general availability. This has consider- ably increased the amount of user generated con- tent present online. This has, however, brought up a few issues. One of the issues is hate speech de- tection, as manual detection has been made nearly impossible by the quantity of data. The only real solution is automated hate speech detection. Our task is detection of hate speech towards immi- grants and women on Twitter (Task A). Hate speech can be defined as ”Any communi- cation that disparages a person or a group on the basis of some characteristic such as race, color, ethnicity, gender, sexual orientation, nationality, religion, or other characteristics.” (Basile et al., 2019) This proves to be a very broad definition, because utterances can be offensive, yet not hate- ful (Davidson et al., 2017). Even manual labeling of hate speech related data is notoriously difficult as hate speech is very subjective in nature (Nobata et al., 2016; Waseem, 2016). The provided dataset consists of collected mes- sages from Twitter in English or Spanish language. Hate speech datasets are very prone to class im- balances (Schmidt and Wiegand, 2017). The pro- vided dataset does not suffer from this problem. The English data contains 10,000 messages with 42.1% of the messages labeled as hate speech. The Spanish data contains 4969 messages and simi- larly to the English part, 41.5% were labeled as hate speech. This gives us a dataset with 14969 messages of which 6270 are categorized as hate- speech. We have not used any additional sources of training data for our models. More informa- tion about the data can be found in the Task defi- nition (Basile et al., 2019). Most research dealing with hate speech has been done in English due to labelled dataset avail- ability. However, this issue is not unique to English-based content. In our work, we explore multilingual approaches, as we recognize data im- balance between languages as one of major chal- lenges of NLP. Multilingual approaches could …

Content truncated to 500 words