KAFK at SemEval-2020 Task 12: Checkpoint Ensemble of Transformers for Hate Speech Classification

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 2023–2029 Barcelona, Spain (Online), December 12, 2020. 2023 KAFK at SemEval-2020 Task 12: Checkpoint Ensemble of Transformers for Hate Speech Classification Kaushik Amar Das 3, Arup Baruah 3, Ferdous Ahmed Barbhuiya 3, Kuntal Dey ♥† 3IIIT Guwahati, India ♥Accenture Technology Labs, Bangalore {kaushikamardas,arup.baruah}@gmail.com, ferdous@iiitg.ac.in, kuntal.dey@accenture.com Abstract This paper presents the approach of Team KAFK for the English edition of SemEval-2020 Task 12. We use checkpoint ensembling to create ensembles of BERT-based transformers and show that it can improve the performance of classification systems. We explore attention mask dropout to mitigate for the poor constructs of social media texts. Our classifiers scored macro-f1 of 0.909, 0.551 and 0.616 for subtasks A, B and C respectively. The code is publicly released online. 1 Introduction The research community has put much effort over the last few years in developing automated detection methods to combat hate speech in social media (Schmidt and Wiegand, 2017). But the complex nature of this phenomenon shows that it has no single solution. The second edition of the OffensEval Workshop (Zampieri et al., 2020), titled OffensEval-2020, is organized with the goal of promoting further research in this domain. The OffensEval-2020 workshop features shared subtasks in five different languages. We participated in the English language which consists of three subtasks (A, B and C). Each subtask is a breakdown of the taxonomy of offensive content. Subtask A (Offensive language identification) is the classification of a post as offensive [OFF] or not offensive [NOT]. Subtask B (Automatic categorization of offence types) determines whether a post containing an insult or a threat is targeted towards an individual, a group, or other [TIN] or simply contains non-targeted profanity and swearing [UNT]. And Subtask C (Offense target identification) is the identification of the target of an offensive post. The targets being individual [IND], group [GRP] and other [OTH]. 2 Related Work Zampieri et al. (2019b) organized OffensEval-2019 where the participants were provided with a dataset of 14,200 annotated tweets (Zampieri et al., 2019a) to perform fine-grained classification of hate speech. HatEval (Basile et al., 2019) is another workshop on the detection of hateful content, specifically, hate speech against women and immigrants in Twitter posts. In this workshop the participants used various approaches such as SVM, sentence and word level embeddings, LSTMs (Hochreiter and Schmidhuber, 1997), Logistic Regression, etc., to build their classification systems, for example, (Baruah et al., 2019; Indurthi et al., 2019). HASOC 2019 (Mandl et al., 2019) was organized with a similar goal, but with a specific interest in Indo-European languages, namely Hindi, English and German. The top-performing systems in HASOC mostly made use of neural network models such as BERT (Devlin et al., 2019), Convolutional Neural Networks (CNN), LSTMs and multilingual embeddings. Other workshops like TRAC (Kumar et al., 2018; Kumar et al., 2020) focused on the identification of aggression in social media text. These workshops featured tasks that consisted of building classifiers that could discriminate between Overtly Aggressive, Covertly Aggressive, …

Content truncated to 500 words