NLP at SemEval-2019 Task 6: Detecting Offensive language using Neural Networks

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 587–592 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 587 NLP at SemEval-2019 Task 6: Detecting Offensive language using Neural Networks Prashant Kapil*, Asif Ekbal* and Dipankar Das+ *Department of Computer Science and Engineering Indian Institute of Technology Patna, India + Department of Computer Science and Engineering Jadavpur University Kolkata, India *{prashant.pcs17, asif }@iitp.ac.in +ddas@cse.jdvu.ac.in Abstract In this paper we built several deep learning architectures to participate in shared task Of- fensEval: Identifying and categorizing Offen- sive language in Social media by semEval- 2019 (Zampieri et al., 2019b). The dataset was annotated with three level annotation schemes and task was to detect between offensive and not offensive, categorization and target iden- tification in offensive contents. Deep learn- ing models with POS information as feature were also leveraged for classification. The three best models that performed best on in- dividual sub tasks are stacking of CNN-Bi- LSTM with Attention, BiLSTM with POS in- formation added with word features and Bi- LSTM for third task. Our models achieved a Macro F1 score of 0.7594, 0.5378 and 0.4588 in Task(A,B,C) respectively with rank of 33rd, 54th and 52nd out of 103, 75 and 65 submis- sions. 1 Introduction Due to the exponential rise in the usage of inter- net user generated content in the form of blogs, posts, comments etc. have been increased mani- fold. Some users also using this platform to tar- get any individual or any particular group on so- cial media on the basis of certain attributes, shar- ing different views. Many studies have been con- ducted on offensive language, hate speech, cyber- bullying, profanity, aggression detection. These contents are major concern for governments, so robust computational systems need to be devel- oped to tackle these posts to maintain social har- mony. This paper is organized as follows. Related work have been discussed in section 2, Methodol- ogy have been described in Section 3 followed by data sets and other settings used to solve the tasks in Section 4. Results and analysis of the models is described in Section 5 with the limitation of the models in Error Analysis in section 6. Section 7 contains the conclusion and Future scope. 1.1 Problem Definition The organizers proposed a hierarchical three level annotation model and divided into three sub tasks. Task A: This task consist of classifying between offensive and not offensive comments Task B: The Offensive language was further needs to be classified into Targeted(TIN) and UnTargeted(UNT). Task C: The targeted offensive needs to be further classified into Individual(IND), Group(GRP) and Other(OTH). 2 Related Work (Nockleby, 2000) defined hate speech as any com- munication that demean any person or any group on the basis of race, color, gender, ethnicity, sex- ual orientation, and nationality. (Kowalski et al., 2014) defined cyber aggression as using digi- tal media to intentionally harm another person. (Schmidt and Wiegand, 2017) presents a survey on the existing research in this field and different …

Content truncated to 500 words