LTL-UDE at SemEval-2019 Task 6: BERT and Two-Vote Classification for Categorizing Offensiveness

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 678–682 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 678 LTL-UDE at SemEval-2019 Task 6: BERT and Two-Vote Classification for Categorizing Offensiveness Piush Aggarwal, Tobias Horsmann, Michael Wojatzki and Torsten Zesch Language Technology Lab University of Duisburg-Essen piush.aggarwal@stud.uni-due.de, {tobias.horsmann, michael.wojatzki, torsten.zesch}@uni-due.de Abstract This paper describes LTL-UDE’s systems for the SemEval 2019 Shared Task 6. We present results for Subtask A and C. In Subtask A, we experiment with an embedding representation of postings and use a Multi-Layer Perceptron and BERT to categorize postings. Our best re- sult reaches the 10th place (out of 103) using BERT. In Subtask C, we applied a two-vote classification approach with minority fallback, which is placed on the 19th rank (out of 65). 1 Introduction The Internet is frequently used for online debates and discussions, where individuals or groups are increasingly often verbally attacked. Online plat- form providers aim to remove such attacking posts or ideally, prevent them from being published. Manual verification of each posting by a human moderator is infeasible due to the high amount of postings created every day. Consequently, auto- mated detection of such attacking postings is the only feasible way to counter this kind of hostility. In this work, we present our results for the SemEval 2019 Shared Task 6: Identifying and Categorizing Offensive Language in Social Me- dia (Zampieri et al., 2019b) on the OLID dataset (Zampieri et al., 2019a). Subtask A focuses on the binary distinction if a post is offensive or not, while Subtask C determines if the target is an indi- vidual, group, or other entity. Our submission for Subtask A ranks 10th, for Subtask C ranks 19th. For Subtask A, we experiment with word list- based classification, using classifiers such as SVM or logistic regression based on sentence embed- dings, and neural network-based models such as a Multi-layer Perceptron (MLP) and Bidirec- tional Encoder Representations from Transform- ers (BERT) (Devlin et al., 2018). We find that the SVM performs best on our development set, but BERT reaches the best result on the test dataset. Moreover, a learning curve experiment suggests that more training data will lead only to minor im- provements. In Subtask C, we choose a two-vote classification approach, where we let two systems compete with a fallback to the minority class in case the systems disagree. This fallback approach has a high robustness between our development and the official test dataset. 2 Related Work Detection of offensive or potentially hurtful on- line postings is investigated under a variety of names. Waseem et al. (2017) focuses on abusive language, Kumar et al. (2018) tackles the prob- lem as aggression while Macbeth et al. (2013) ap- proaches this problem as cyberbullying to mention just a few. Furthermore, the field of hate speech detection is strongly related, which aims at detect- ing a similar kind of online statements (Waseem and Hovy, 2016; Wojatzki et al., 2018). Common approaches to detecting such socially …

Content truncated to 500 words