PUM at SemEval-2020 Task 12: Aggregation of Transformer-based Models’ Features for Offensive Language Recognition

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1615–1621 Barcelona, Spain (Online), December 12, 2020. 1615 PUM at SemEval-2020 Task 12: Aggregation of Transformer-based models’ features for offensive language recognition Piotr Janiszewski, Mateusz Skiba and Urszula Wali´nska Faculty of Computing and Telecommunications Poznan University of Technology Poznan, Poland {1piotr.janiszewski,xmateusz.skiba, urszula.walinska96}@gmail.com Abstract In this paper, we describe the PUM team’s entry to the SemEval-2020 Task 12. Creating our solu- tion involved leveraging two well-known pretrained models used in natural language processing: BERT and XLNet, which achieve state-of-the-art results in multiple NLP tasks. The models were fine-tuned for each subtask separately and features taken from their hidden layers were combined and fed into a fully connected neural network. The model using aggregated Transformer features can serve as a powerful tool for offensive language identification problem. Our team was ranked 7th out of 40 in Sub-task C - Offense target identification with 64.727% macro F1-score and 64th out of 85 in Sub-task A - Offensive language identification (89.726% F1-score). 1 Introduction Nowadays, abusive content spreading in social media poses a serious problem. The offensive language is pervasive and due to the users’ anonymity on the Internet, people feel unreachable when it comes to the punishment. As a result, many people may suffer from being insulted. Multiple social media platforms and companies noticed the emerging problem and have already investigated possible solutions. One of them is to use computational methods to detect hate speech in various types of utterances posted on the web, what has already drawn considerable attention as evidenced in recent publications (Davidson et al., 2017), (Malmasi and Zampieri, 2017), (Wiegand et al., 2018). Nevertheless, the ability of identifying offense automatically is the first step which has to be undertaken. In this paper we tackle the problem of Identifying and Categorizing Offensive Language in Social Media. It is the 12-th task of SemEval-2020 competition (Zampieri et al., 2020). This is also the second edition of OffensEval, which started in 2019 (Zampieri et al., 2019b). The problem is to detect and assign offensive language occurrences to various pre-defined categories using Natural Language Processing methods. A dedicated datasets consisting of tweets crawled from Twitter were created specifically for this task. Organizers of the competition provided the participants also with the dataset from previous year’s edition – OLID (Zampieri et al., 2019a), which we used during the development of our solution. Each example is appropriately labelled according to a specific subtask: • Sub-task A – Identifying if an utterance is offensive (OFF) or not (NOT), • Sub-task B – Categorizing if a tweet is a targeted (TIN) or untargeted (UNT) insult or threat, • Sub-task C – Identifying if the offensiveness target is individual (IND) or group (GRP) or other (OTH) e.g. a situation or an event. OffensEval 2020 features a multilingual dataset with five languages. The languages included are: Arabic, Danish, English, Greek and Turkish (Mubarak et al., 2020), (Sigurbergsson and Derczynski, 2020), (Rosenthal et al., 2020), (Pitenis et al., 2020), (C¸ …

Content truncated to 500 words