TüKaSt at SemEval-2019 Task 6: Something Old, Something Neu(ral): Traditional and Neural Approaches to Offensive Text Classification

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 763–769 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 763 T¨uKaSt at SemEval-2019 Task 6: Something Old, Something Neu(ral): Traditional and Neural Approaches to Offensive Text Classification Madeeswaran Kannan Lukas Stein Department of Linguistics University of T¨ubingen, Germany {mkannan,lstein}@sfs.uni-tuebingen.de Abstract We describe our system (T¨uKaSt) submitted for Task 6: Offensive Language Classifica- tion, at SemEval 2019. We developed multi- ple SVM classifier models that used sentence- level dense vector representations of tweets enriched with sentiment information and term weighting. Our best results achieved F1 scores of 0.734, 0.660 and 0.465 in the first, second and third sub-tasks respectively. We also de- scribe a neural network model that was devel- oped in parallel but not used during evaluation due to time constraints. 1 Introduction We live in a day and age where social media pervades through every aspect of lives. Constant growth of social media platforms and rapid exposure to them are changing how human communication is perceived and working (Sticca and Perren, 2013). As much as social media and the general web help its users stay connected and informed, misuse of these channels grows with them. As long as anonymity continues to play a central role in online interactions, one must contend with individuals and entities who feel empowered to behave aggressively for that very reason (Burnap and Williams, 2015). Recent events have forced social media platforms to take a renewed interest in limiting the negative influence of such users for commercial and practical reasons. However, the sheer size of data on social media makes it impossible to manually observe, thereby calling the creation of systems that automatically detect potentially offensive content. The SemEval 2019 - OffensEval Shared Task (Zampieri et al., 2019b) is aiming exactly at that need, calling for system able to detect offensive language in social media data. The task is split into three sub-tasks: Detecting offensive language, de- termining whether it is offensive directly towards a target, and target specification. This paper fol- lows our approach of applying both traditional and more sophisticated neural models to the task of of- fensive text classification. 2 Preprocessing All tweets were converted to lowercase and to- kenised. Stop-words were removed uncondition- ally, but hashtags and user mentions were removed during the training of certain model variants. 3 Training Data The SVM models were trained and evaluated exclusively on the training data provided by Zampieri et al. (2019a) with ten-fold cross- validation. For the RNN models, we also used the trail data that was provided before the start of the training phase. The samples were shuffled before each training run, and a 80-20 split was performed to generate training and validation sets. 300-dimensional, pre-trained FastText embed- dings (Mikolov et al., 2018) were used to train the SVM models. The FastText embeddings were chosen due to their subword-information (Bojanowski et al., 2017), making it easier to deal with social media data, which is prone to contain spelling …

Content truncated to 500 words