Ghmerti at SemEval-2019 Task 6: A Deep Word- and Character-based Approach to Offensive Language Identification

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 617–621 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 617 Ghmerti at SemEval-2019 Task 6: A Deep Word- and Character-based Approach to Offensive Language Identification Ehsan Doostmohammadi♣, ♠, Hossein Sameti♣, Ali Saffar♠ ♣Speech Processing Lab, Department of Computer Engineering, Sharif University of Technology, Tehran, Iran ♠NazarBin, Tehran, Iran e.doostm72@student.sharif.edu, sameti@sharif.edu, saffar@nazarbin.com Abstract This paper presents the models submitted by Ghmerti team for subtasks A and B of the Of- fensEval shared task at SemEval 2019. Offen- sEval addresses the problem of identifying and categorizing offensive language in social me- dia in three subtasks; whether or not a content is offensive (subtask A), whether it is targeted (subtask B) towards an individual, a group, or other entities (subtask C). The proposed ap- proach includes character-level Convolutional Neural Network, word-level Recurrent Neural Network, and some preprocessing. The per- formance achieved by the proposed model for subtask A is 77.93% macro-averaged F1-score. 1 Introduction The massive rise in user-generated web content, alongside with the freedom of speech in social me- dia and anonymity of the users has brought about an increase in online offensive content and anti- social behavior. The consequences of such behav- ior on genuine users of the social media have be- come a serious concern for researchers in Natural Language Processing and related fields in recent years. The shared task number 6 at SemEval 2019, OffensEval (Zampieri et al., 2019b), proposes to model the task of offensive language identification hierarchically, which means identifying the offen- sive content, whether it is targeted, and if so, the target of the offense. In OffensEval, offensive lan- guage is defined as “any form of non-acceptable language (profanity) or a targeted offense, which can be veiled or direct” which includes “insults, threats, and posts containing profane language or swear words” (Zampieri et al., 2019b). We have participated in the first two subtasks (A and B) of OffensEval with the proposed approach of a deep model consisting of a Recurrent Neural Network (RNN) for word-level and Convolutional Neural Network (CNN) for character-level pro- cessing1. Character-level processing is beneficial, as offensive comments are likely to follow un- orthodox writing styles, contain obfuscated words, or have irregular word separation which leads to tokenization issues (Mehdad and Tetreault, 2016; Nobata et al., 2016). We also experimented with two other methods, a Support Vector Machine (SVM) with TFIDF and count features and another SVM with BERT (Devlin et al., 2018) -encoded sentences as input, both with lower performances comparing with the deep model. After overviewing the related work in section 2, we discuss the methodology and the data in details in section 3, and the results in section 4. In section 5, we analyze the results and conclude the paper in section 6. 2 Related Work Offensive language identification has been of in- terest for researchers in recent years. Early work in the related fields include detection of online trolling (Cambria et al., 2010), racism (Greevy …

Content truncated to 500 words