jhan014 at SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 652–656 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 652 jhan014 at SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media Jiahui Han University of Ottawa jhan014@uottawa.ca Shengtan Wu Jackson State University shengtan.wu @students.jsums.edu Xinyu Liu Purdue University liu1957@purdue.edu Abstract In this paper, the team jhan014 presents two methods to identify and categorize the offen- sive language in Twitter. In the first method, we develop a deep neural network consisting of bidirectional recurrent layers with Gated Recurrent Unit (GRU) cells and fully con- nected layers. In the second method, we estab- lish a probabilistic model, modified sentence offensiveness calculation (MSOC) to evalu- ate the sentence offensiveness level and target level according to different sub-tasks. Based on task results, We evaluate the performance of each method based on F1 score and analyze the advantages and disadvantages of these two methods with the type I error and type II error. In conclusion, deep neural network behaves well in all subtasks but has more type I er- ror and fails to categorize subclasses with mi- nor data or less character, while MSOC model does better in target categorizing but has more type II error in offensive identifying. 1 Introduction With the popularity of social media like Twitter, offensive language has become a serious prob- lem(Zampieri et al., 2019b) on these media plat- forms. People have to face with abusive behav- ior from others in social media from time to time. To solve this problem, finding a method to iden- tify and categorize offensive languages is an ur- gent need. In this paper, two different methods, deep learning method and modified sentence offensive- ness calculation method, are used to categorize the type and target of offensive language and the difference of results are revealed and analyzed. 2 Related Work Deep learning method: Deep learning methods are widely used in natural language processing (Liu et al., 2016). Models like Recursive neural network are commonly used to identify if a sentence contain certain emotion. In our work, a deep neural network with GRU layers and all connection layers is built. Offensiveness Content Filtering: Offensive lan- guage targets can be understood through the sen- tence structure (Silva et al., 2016) or lexical analy- sis (ElSherief et al., 2018). We take both sentence structure and the offensiveness level of words into consideration. Furthermore, we also concentrate on the special punctuation (like @ and # ) in on- line social media. 3 Methodology and Data 3.1 Deep Neural Network In the offensive language detection task, we de- veloped a deep neural network based system with binary cross-entropy output. System Design The system consists of bidi- rectional recurrent layers with Gated Recur- rent Unit(GRU) cells and fully connected layers (Chung et al., 2014). Because the output of the last time-step is used as the embedding of a sen- tence, we conduct zero padding in the beginning of each sequence to construct the feature matrix. …

Content truncated to 500 words