JCTDHS at SemEval-2019 Task 5: Detection of Hate Speech in Tweets using Deep Learning Methods, Character N-gram Features, and Preprocessing Methods

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 426–430 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 426 Abstract In this paper, we describe our submissions to SemEval-2019 contest. We tackled subtask A - “a binary classification where systems have to predict whether a tweet with a given target (women or immigrants) is hateful or not hateful”, a part of task 5 “Multilingual detection of hate speech against immigrants and women in Twitter (HatEval)”. Our system JCTDHS (Jerusalem College of Technology Detects Hate Speech) was developed for tweets written in English. We applied various supervised ML methods, various combinations of n-gram features using the TF-IDF scheme. In addition, we applied various combinations of eight basic preprocessing methods. Our best submission was a special bidirectional RNN, which was ranked at the 11th position out of 68 submissions. 1 Introduction Hate Speech is usually defined as communication that contains contempt or hatred towards a person or a group of people on the basis of some characteristic e.g., color, ethnicity, gender, nationality, race, religion, and sexual orientation. The phenomenon of hate speech in social media has grown in recent years (Eadicicco. 2014). A strong connection between hate speech and actual hate crimes has been shown in Watch (2014). In light of the huge amount of information in social media, early detection of people using hate speech could prevent them from carrying out their hate speech. Therefore, the detection of hate speech in social media has become an issue of increasing importance (Moulson. 2016). In this paper, we describe our six models (each model with another team member as the first author) submitted to task 5-A for tweets written in English. The full description of this task is given in Basile et al. (2019). The structure of the rest of the paper is as follows. Section 2 introduces a background concerning hate speech, tweet classification, and data preprocessing. Section 3 presents, in general, the description of Task 5. In Section 4, we describe the submitted models and their experimental results. Section 6 summarizes and suggests ideas for future research. 2 Background 2.1 Hate Speech Waseem and Hovy (2016) introduced a list of criteria founded in critical race theory and used them to label a publicly available corpus of more than 16k tweets with tags about both racial and sexist offenses. Nobata et al. (2016) developed a machine learning based method to detect hate speech on online user comments from two domains. They also built a corpus of user comments annotated accordingly to three subcategories (hate speech, derogatory, profanity). Schmidt and Wiegand (2017) introduced a survey of the NLP methods that were developed in order to detect hate speech. Davidson et al. (2017) presented a multi-class classifier to distinguish between three categories: hate speech, offensive language, and none of these two. The analysis of the predictions and the errors shows when they can reliably separate hate speech from other types of offensive language (e.g., tweets with the highest predicted …

Content truncated to 500 words