Nikolov-Radivchev at SemEval-2019 Task 6: Offensive Tweet Classification with BERT and Ensembles

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 691–695 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 691 Nikolov-Radivchev at SemEval-2019 Task 6: Offensive Tweet Classification with BERT and Ensembles Victor Radivchev Sofia University, FMI victor.radivchev@gmail.com Alex Nikolov Sofia University, FMI alexnickolow@gmail.com Abstract This paper examines different approaches and models towards offensive tweet classification which were used as a part of the OffensE- val 2019 competition. It reviews Tweet pre- processing, techniques for overcoming unbal- anced class distribution in the provided test data, and comparison of multiple attempted machine learning models. 1 Introduction The purpose of this paper is to explore different approaches towards classifying tweets based on whether they are offensive or not, whether offensive tweets are targeted, and identifying the target group of offensive tweets either an individual, a group, or other. Those are the terms of the OffensEval 2019 competition in which we participated. Each of the described activities constituted a separate subtask from the competition. A maximum of three submissions were allowed per subtask which required careful preliminary analysis of the model results during the training phase. A training set of over 13,000 tweets, containing labels for all three subtasks. Each of the subtasks was scored using macro F1 score. 2 Related Work One of the most effective strategies for tackling this problem is to use computational methods to identify offense, aggression, and hate speech in user-generated content (e.g. posts, comments, mi- croblogs, etc.). This topic has attracted significant attention recently as evidenced in publications from the last two years. Survey papers describing key areas that have been explored for this task include (Schmidt and Wiegand, 2017), (Fortuna and Nunes, 2018) and (Malmasi and Zampieri, 2017). The dataset for this competition is explained in (Zampieri et al., 2019a) and different approaches to the same problem are reported in (Zampieri et al., 2019b). In order to classify correctly abusive language it is important to analyze its types. A proposal of typology of abusive language sub-tasks is presented in (Waseem et al., 2017) and (ElSherief et al., 2018) examines the target of the speech: either directed towards a specific person or entity, or generalized towards a group of people sharing a common protected characteristic. (Fiˇser et al., 2017) proposes a legal framework, dataset and annotation schema of socially unacceptable dis- course practices on social networking platforms in Slovenia. Finally, a recent discussion on identifying profanity vs. hate speech is presented in (Malmasi and Zampieri, 2018). This work highlighted the challenges of distinguishing be- tween profanity, and threatening language which may not actually contain profane language. Approaches to detecting hate speech on Twit- ter using convolutional neural networks and convolution-GRU based deep neural network are discussed in (Gamb¨ack and Sikdar, 2017) and (Zhang et al., 2018) respectively. Additional related work is presented in work- shops such as TA-COS1, Abusive Language Online2, and TRAC3 and related shared tasks such as GermEval (Wiegand et al., 2018) and TRAC (Kumar et al., 2018). 1http://ta-cos.org/ 2https://sites.google.com/site/ abusivelanguageworkshop2017/ …

Content truncated to 500 words