SU-NLP at SemEval-2020 Task 12: Offensive Language IdentifiCation in Turkish Tweets

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 2171–2176 Barcelona, Spain (Online), December 12, 2020. 2171 SU-NLP at SemEval-2020 Task 12: Offensive Language Identification in Turkish Tweets Anıl ¨Ozdemir Sabancı University ˙Istanbul, Turkey aozdemir@sabanciuniv.edu Reyyan Yeniterzi Sabancı University ˙Istanbul, Turkey reyyan@sabanciuniv.edu Abstract This paper summarizes our group’s efforts in the offensive language identification shared task, which is organized as part of the International Workshop on Semantic Evaluation (Sem-Eval2020). Our final submission system is an ensemble of three different models, (1) CNN-LSTM, (2) BiLSTM-Attention and (3) BERT. Word embeddings, which were pre-trained on tweets, are used while training the first two models. BERTurk, which is the first BERT model for Turkish, is also explored. Our final submitted approach ranked as the second best model in the Turkish sub-task. 1 Introduction With the growing popularity of social media all over the world, more and more people are using these platforms, and unfortunately, many people regularly face offensive language in these environments. Such offensive language can have devastating effects on users depending on their ages and psychological conditions (O’Keeffe et al., 2011). In the current big data era, manual filtering of such content is not possible; therefore, development of effective automated solutions is a necessity. This important task of identifying the offensive language has been supported by SemEval organizers since last year, as part of the OffensEval task (Zampieri et al., 2019). Last year’s task focused only on English and this year on its second run, the organizers aimed for a multilingual task and run the challenge for five languages, Arabic, Danish, English, Greek and Turkish (Zampieri et al., 2020). This year we specifically focused on Turkish, in which a high-performance automated system is very much needed, due to the widespread use of social media by Turkish-speaking people of all ages. Within the framework of this problem, we initially analyzed different state-of-the-art neural network models and different word embeddings. Our final system consists of a weighted ensemble model combining the following 3 models; Convolutional Neural Network (CNN) with Long Short Term Memory (LSTM), Bidirectional Long Short Term Memory (BiLSTM) with Attention and the recently released Turkish BERT model. Additionally, word embeddings which are trained over Turkish tweets are explored for better domain and task adaptation, and for this particular task they provide much better performance compared to other available pre-trained embeddings. With our proposed system we achieved the 2nd place in Turkish sub-task. Our code is available online1. 2 Task and Data Description OffensEval task consists of three subtasks. This year for the four new languages, the organizers organize only the first sub-task A, which is the offensive language identification task. The goal of this sub-task is to predict whether a given content is offensive or not. In the task, a post is described as offensive if it includes insults, threats, profane language or swear words (Zampieri et al., 2019). The data collection created by C¸ ¨oltekin (2020), is used for OffensEval Turkish task. The collection consists of 34805 Turkish tweets …

Content truncated to 500 words