UHH-LT at SemEval-2019 Task 6: Supervised vs. Unsupervised Transfer Learning for Offensive Language Detection

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 782–787 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 782 UHH-LT at SemEval-2019 Task 6: Supervised vs. Unsupervised Transfer Learning for Offensive Language Detection Gregor Wiedemann Eugen Ruppert* Chris Biemann Language Technology Group / *Base.Camp Department of Informatics University of Hamburg, Germany {gwiedemann, ruppert, biemann}@informatik.uni-hamburg.de Abstract We present a neural network based approach of transfer learning for offensive language de- tection. For our system, we compare two types of knowledge transfer: supervised and unsu- pervised pre-training. Supervised pre-training of our bidirectional GRU-3-CNN architecture is performed as multi-task learning of parallel training of five different tasks. The selected tasks are supervised classification problems from public NLP resources with some over- lap to offensive language such as sentiment detection, emoji classification, and aggressive language classification. Unsupervised transfer learning is performed with a thematic cluster- ing of 40M unlabeled tweets via LDA. Based on this dataset, pre-training is performed by predicting the main topic of a tweet. Results indicate that unsupervised transfer from large datasets performs slightly better than super- vised training on small ‘near target category’ datasets. In the SemEval Task, our system ranks 14 out of 103 participants. 1 Introduction The automatic detection of hate speech, cyberbul- lying, abusive, aggressive or offensive language has become a vital field of research in natural lan- guage processing (NLP) during recent years. Es- pecially in social media, the tone of conversations escalates in a disturbing way that often threatens a free, democratic and argumentative discourse of users. Concerning the tremendous amount of digi- tal texts posted on platforms such as Twitter, Face- book or in comments sections of online newspa- pers, automatic approaches to offensive language detection are of high relevancy for moderation and filtering of content as well as for studying the phe- nomenon of offensive language use in social me- dia at large scale. To take account of this development, a shared task on ”offensive language detection” was con- ducted at the SemEval 2019 workshop. This paper describes our approach to the shared task 6 (Of- fensEval) as organized and described in detail by Zampieri et al. (2019b). The task contains three hi- erarchically ordered sub-tasks: Task A requires a classification of tweets into either offensive (OFF) or not offensive (NOT), Task B subdivides all of- fensive tweets into either targeted insults (TIN) or generally offensive expressions not targeted to any specific entity (UNT), and Task C finally asks to assign one out of three specific target labels to all targeted insults: groups (GRP, e.g. ethnic groups), individuals (IND, e.g. a politician or a specific Twitter user), or other (OTH, e.g. the media in- dustry). The dataset consisting of 14,100 exam- ples (13,240 in the training set, 860 in the test set) was annotated via crowdsourcing (Zampieri et al., 2019a). Each tweet was labeled by at least two an- notators who must reach an agreement of at least 66% for including the tweet in the dataset. The dataset …

Content truncated to 500 words