KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 2054–2059 Barcelona, Spain (Online), December 12, 2020. 2054 KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media Ali Safaya, Moutasem Abdullatif, Deniz Yuret KUIS AI Lab Koc¸ University, ˙Istanbul/Turkey {asafaya19,mabdullatif18,dyuret}@ku.edu.tr Abstract In this paper, we describe our approach to utilize pre-trained BERT models with Convolutional Neural Networks for sub-task A of the Multilingual Offensive Language Identification shared task (OffensEval 2020), which is a part of the SemEval 2020. We show that combining CNN with BERT is better than using BERT on its own, and we emphasize the importance of utilizing pre-trained language models for downstream tasks. Our system, ranked 4th with macro averaged F1-Score of 0.897 in Arabic, 4th with score of 0.843 in Greek, and 3rd with score of 0.814 in Turkish. Additionally, we present ArabicBERT, a set of pre-trained transformer language models for Arabic that we share with the community. 1 Introduction Recently, we have observed an increase in social media usage and a similar increase in hateful and offen- sive speech. Solutions to this problem vary from manual control to rule-based filtering systems; however, these methods are time-consuming or prone to errors if the full context is not taken into consideration while assessing the sentiment of the text (Saif et al., 2016). In Subtask-A of the shared task of Multilingual Offensive Language Identification (OffensEval2020), we focus on detecting offensive language on social media platforms, more specifically, on Twitter. The organizers provided data from five different languages, which we worked on three languages of them, namely, Arabic (Mubarak et al., 2020), Greek (Pitenis et al., 2020), and Turkish (C¸ ¨oltekin, 2020). More details about the annotation process have been described in task description paper (Zampieri et al., 2020). The approach used combines the knowledge embedded in pre-trained deep bidirectional transformer BERT (Devlin et al., 2019) with Convolutional Neural Networks (CNN) for text (Kim, 2014), which is one of the most utilized approaches for text classification tasks. This combination of models has been shown to yield better results than using BERT or CNN on their own, as was shown in (Li et al., 2019), and shown in this paper. This model, and with minimum text pre-processing, ranked 4th in Arabic, 4th in Greek, and 3rd in Turkish among more than 40 participants. In the following sections of this paper, previous work is mentioned in Section 2, next, the data is described in Section 3, then the details of the model and the other experiments are described in Section 4. Finally, the submissions and the other experiments are detailed in Section 5. 2 Background Extensive work has been performed to solve the task of offensive speech identification, which classifies among text classification tasks. Approaches to solve this problem vary from using lexical resources, lin- guistic features, and meta information (Schmidt and Wiegand, 2017), to machine learning (ML) models This work is licensed under a Creative Commons Attribution 4.0 International License. License details: http:// creativecommons.org/licenses/by/4.0/. Our source …

Content truncated to 500 words