TheNorth at SemEval-2020 Task 12: Hate Speech Detection Using RoBERTa

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 2197–2202 Barcelona, Spain (Online), December 12, 2020. 2197 TheNorth at SemEval-2020 Task 12: Hate Speech Detection using RoBERTa Pedro Alonso Rajkumar Saini EISLAB Machine Learning Lule˚a University of Technology Lule˚a, Sweden {pedro.alonso,rajkumar.saini,gyorgy.kovacs}@ltu.se Gy¨orgy Kov´acs Abstract Hate speech detection on social media platforms is crucial as it helps to avoid severe harm to marginalized people and groups. The application of Natural Language Processing (NLP) and Deep Learning has garnered encouraging results in the task of hate speech detection. The expression of hate, however, is varied and ever-evolving. Thus better detection systems need to adapt to this variance. Because of this, researchers keep on collecting data and regularly come up with hate speech detection competitions. In this paper, we discuss our entry to one such competition, namely the English version of sub-task A for the OffensEval competition. Our contribution can be perceived through our results, that was first an F1-score of 0.9087, and with further refinements described here climb up to 0.9166. It serves to give more support to our hypothesis that one of the variants of BERT, namely RoBERTa can successfully differentiate between offensive and non-offensive tweets, given the proper preprocessing steps. 1 Introduction The present paper is a detailed description of the system1 we use in the competition of detecting hate speech (Zampieri et al., 2020). The system proposed here was designed for the English language portion of sub-task A. We have decided to investigate a Deep Learning model, due to the considerable amount of data we had available at our disposal. The macro F1-score of 0.9087 attained by our model in the competition puts us close to the top half of the competitors. While this can be discouraging, it should be noted that only one million samples were used to train our models due to the limitations of the available hardware capacity. Furthermore, even with the above-mentioned limitation, the difference between the classification score we attained, and the top team’s classification score was only 0.02. This suggests that with some improvements (e.g. optimization of hyper-parameters and samples used), our system can be capable of State-of-the-art performance. 1.1 Related work Delving into hate speech detection, we were able to gather previous research done on the topic, some by using Convolutional Neural Networks (CNNs) like (Zhang et al., 2018), (Gamb¨ack and Sikdar, 2017) and (Pedro Alonso, 2019). As quite many research papers have dealt with hate speech with CNN, we decided to experiment with a different deep learning model that has shown excellent results in NLP tasks, namely BERT. To start, we found (Mozafari et al., 2019), which dealt with a task similar to ours, with good results. Therefore, we decided to try some of BERT’s variants, namely DistilBERT (Sanh et al., 2019a) and RoBERTa (Liu et al., 2019a). After reading throughout the literature, we consider that the use of RoBERTa and DistilBERT, has not been researched enough, and we consider that our experiments serve to give some more visibility to …

Content truncated to 500 words