Embeddia at SemEval-2019 Task 6: Detecting Hate with Neural Network and Transfer Learning Approaches

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 604–610 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 604 Embeddia at SemEval-2019 Task 6: Detecting Hate with Neural Network and Transfer Learning Approaches Andraˇz Pelicon, Matej Martinc, Petra Kralj Novak Joˇzef Stefan Institute, Jamova 39, 1000 Ljubljana, Slovenia {Andraz.Pelicon, Matej.Martinc, Petra.Kralj.Novak}@ijs.si Abstract SemEval-2019 Task 6 was OffensEval: Iden- tifying and Categorizing Offensive Language in Social Media. The task was further divided into three sub-tasks: offensive language iden- tification, automatic categorization of offense types, and offense target identification. In this paper, we present the approaches used by the Embeddia team, who qualified as fourth, eigh- teenth and fifth on the three sub-tasks. A dif- ferent model was trained for each sub-task. For the first sub-task, we used a BERT model fine-tuned on the provided dataset, while for the second and third tasks we developed a cus- tom neural network architecture which com- bines bag-of-words features and automatically generated sequence-based features. Our re- sults show that combining automatically and manually crafted features fed into a neural architecture outperform transfer learning ap- proach on more unbalanced datasets. 1 Introduction Over the years, computer-mediated communica- tion, like the one on social media, has become one of the key ways people communicate and share opinions. Computer-mediated communica- tion differs in many ways, both technically and culturally, from more traditional communication technologies (Kiesler et al., 1984). However, the ability to fully or partially hide our identity behind an internet persona leads people to type things they would never say to someone’s face (Shaw, 2011). Not only is hate speech more likely to happen on the Internet, where anonymity is eas- ily obtained and speakers are psychologically dis- tant from their audience, but its online nature also gives it a far-reaching and determinative impact (Shaw, 2011). Although most forms of intolerance are not criminal, hate speech and other speech acts designed to harass and intimidate (rather than merely express criticism or dissent), deteriorate public discourse and opinions, which can lead to a more radicalized society. Online communities, social media platforms, and technology companies have been investing heavily in ways to cope with offensive language to prevent abusive behavior in social media. Social media companies Facebook, Twitter and Google’s YouTube have greatly accelerated their removal of online hate speech, and report reviewing over two-thirds of complaints within 24 hours. It has been proven in practice that naive word filtering systems do not manage to scale well to different forms of hate and aggression (Schmidt and Wie- gand, 2017). The most promising strategy for de- tecting abusive language is to use advanced com- putational methods. This topic has attracted sig- nificant attention in recent years as evidenced in recent publications (Waseem et al., 2017; David- son et al., 2017; Malmasi and Zampieri, 2018). The SemEval-2019 Task 6 — OffensEval: Iden- tifying and Categorizing Offensive Language in Social Media (Zampieri et al., 2019b) is to use machine learning text classification methods to identify offensive content …

Content truncated to 500 words