DA-LD-Hildesheim at SemEval-2019 Task 6: Tracking Offensive Content with Deep Learning using Shallow Representation

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 577–581 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 577 DA-LD-Hildesheim at SemEval-2019 Task 6: Tracking Offensive Content with Deep Learning using Shallow Representation Sandip Modha DA-IICT Gandhinagar India sjmodha @gmail.com Prasenjit Majumder DA-IICT Gandhinagar India p majumder @daiict.ac.in Thomas Mandl Uni. of Hildesheim Hildesheim Germany mandl @uni-hildesheim.de Daksh Patel LDRP-ITR Gandhinagar India dakshpatel68 @gmail.com Abstract This paper presents the participation of team DA-LD-Hildesheim of Information Retrieval and Language Processing lab at DA-IICT, In- dia in Semeval-19 OffenEval track. The aim of this shared task is to identify offensive con- tent at fined-grained level granularity. The task is divided into three sub-tasks. The system is required to check whether social media posts contain any offensive or profane content or not, targeted or untargeted towards any entity and classifying targeted posts into the individ- ual, group or other categories. Social media posts suffer from data sparsity problem, There- fore, the distributed word representation tech- nique is chosen over the Bag-of-Words for the text representation. Since limited labeled data was available for the training, pre-trained word vectors are used and fine-tuned on this clas- sification task. Various deep learning models based on LSTM, Bidirectional LSTM, CNN, and Stacked CNN are used for the classifica- tion. It has been observed that labeled data was highly affected with class imbalance and our technique to handle the class-balance was not effective, in fact performance was degraded in some of the runs. Macro F1 score is used as a primary evaluation metric for the performance. Our System achieves Macro F1 score = 0.7833 in sub-task A, 0.6456 in the sub-task B and 0.5533 in the sub-task C. 1 Introduction NLP researchers are developing innovative sys- tems based on the input of the text data. The power of predictions has moved from simple sen- timent classification task to much more advanced labeling of the content. The task related to hate, aggression, abusive or offensive speech currently attracts research more to algorithms making deci- sions which can also be ambiguous for humans. Due to the availability of standard datasets, such data collections are created based on social media data and are offered at forum like TRAC 1 (Kumar et al., 2018), GermEval 2, and SemEval OffenEval 2019 (Zampieri et al., 2019a)3. The exponential rise in social media user-base backed by the cutting edge mobile data technolo- gies leads to the inorganic growth in the posts related to hate speech or offensive speech. Re- searchers working in the area of domain-specific sentiment analysis move to the problem of domain specific or open domain hate or offensive speech detection. They are reshaping the hate speech problem into the new notion like abusive, aggres- sive, or offensive speech. Such categorization of social media posts, help law-enforcement agencies with the surveillance of the social media. The shared task in SemEval-OffenEval 2019 was introduced as a 3-level classification task (Zampieri et al., 2019b). In the first level, sub-task A, …

Content truncated to 500 words