HHU at SemEval-2019 Task 6: Context Does Matter - Tackling Offensive Language Identification and Categorization with ELMo

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 628–634 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 628 HHU at SemEval-2019 Task 6: Context Does Matter - Tackling Offensive Language Identification and Categorization with ELMo Alexander Oberstrass1 Julia Romberg1 Anke Stoll2 Sefan Conrad1 1Institute of Computer Science, Heinrich Heine University D¨usseldorf, Germany alexander.oberstrass@hhu.de {romberg,conrad}@cs.uni-duesseldorf.de 2Department of Social Sciences, Heinrich Heine University D¨usseldorf, Germany anke.stoll@hhu.de Abstract We present our results for OffensEval: Iden- tifying and Categorizing Offensive Language in Social Media (SemEval 2019 - Task 6). Our results show that context embeddings are important features for the three different sub- tasks in connection with classical machine and with deep learning. Our best model reached place 3 of 75 in sub-task B with a macro F1 of 0.719. Our approaches for sub-task A and C perform less well but could also deliver promising results. 1 Introduction User generated content in social media platforms such as Twitter often includes high levels of rude, offensive, or sometimes even hateful language. The increasing vulgarity in online discussions and user comment sections have recently been dis- cussed as relevant issues in society as well as in science. The identification of offensiveness, aggression, and hate speech in user-generated content has been addressed in recent research (Waseem et al., 2017; Davidson et al., 2017a; Malmasi and Zampieri, 2018) and previous shared tasks (Wiegand et al., 2018; Kumar et al., 2018). However, detect- ing such content automatically is still challenging. We developed classification models to identify of- fensive language, different categories of offense types, and targets of offensive language through- out the SemEval-2019 challenge on Identifying and Categorizing Offensive Language in Social Media (Zampieri et al., 2019b). 2 Methodology and Data In this section, we introduce the datasets, the fea- tures and classifiers we use. 2.1 Datasets The training dataset provided for this task is fur- ther described in Zampieri et al. (2019a). For sub-task A: Offensive Language Detection 4.400 offensive (OFF) and 8.840 not offensive (NOT) tweets are given for the training and 240 offensive plus 620 not offensive tweets are given as test data. Sub-task B: Categorization of Offensive Language is provided with a training set of 3876 targeted in- sult (TIN) and 524 untargeted (UNT) tweets and a test set of 213 targeted insult plus 27 untargeted tweets. The train data distribution for sub-task C: Offensive Language Target Identification are 2407 tweets targeting an individual (IND), 1074 target- ing a group (GRP) and 395 targeting any other cat- egory (OTH). The test data given counts 100, 78 and 35, respectively. We use Davidson et al. (2017b)’s dataset for hate speech detection as an additional source of knowledge and to address the problem of overfit- ting in our deep learning models. Tweets were col- lected and filtered using a lexicon of common hate speech terms. The remaining tweets were then each classified by at least 3 CrowdFlower users into the three categories hate speech, offensive lan- guage or none. Each tweet …

Content truncated to 500 words