Fermi at SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media using Sentence Embeddings

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 611–616 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 611 Fermi at SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media using Sentence Embeddings Vijayasaradhi Indurthi1,3, Bakhtiyar Syed1, Manish Shrivastava1 Manish Gupta1,2, Vasudeva Varma1 1 IIIT Hyderabad, 2 Microsoft, 3 Teradata 1{vijaya.saradhi, syed.b}@research.iiit.ac.in 1{m.shrivastava, manish.gupta, vv}@iiit.ac.in 2gmanish@microsoft.com 3vijayasaradhi.indurthi@teradata.com Abstract This paper describes our system (Fermi) for Task 6: OffensEval: Identifying and Cate- gorizing Offensive Language in Social Me- dia of SemEval-2019. We participated in all the three sub-tasks within Task 6. We evalu- ate multiple sentence embeddings in conjunc- tion with various supervised machine learn- ing algorithms and evaluate the performance of simple yet effective embedding-ML combi- nation algorithms. Our team (Fermi)’s model achieved an F1-score of 64.40%, 62.00% and 62.60% for sub-task A, B and C respectively on the official leaderboard. Our model for sub- task C which uses pretrained ELMo embed- dings for transforming the input and uses SVM (RBF kernel) for training, scored third position on the official leaderboard. Through the paper we provide a detailed de- scription of the approach, as well as the results obtained for the task. 1 Introduction Social media provides anonymity which can be misused to target offensive comments to targeted parties. Users may engage in generating offen- sive content on social media which may show aggressive behaviour and may also include hate speech. As a result, it is imperative for social media platforms to invest heavily in creating so- lutions which can identify offensive language and to prevent such behaviour on social media. Using computational methods to identify of- fense, aggression and hate speech in user gener- ated content has been gaining attention in the re- cent years as evidenced in (Waseem et al., 2017; Davidson et al., 2017; Malmasi and Zampieri, 2017; Kumar et al., 2018) and workshops such as Abusive Language Workshop (ALW) 1 and Work- 1https://sites.google.com/view/alw2018 shop on Trolling, Aggression and Cyberbullying (TRAC) 2. 2 Related Work In this section we briefly describe other work in this area. Papers published in the last two years include the surveys by (Schmidt and Wiegand, 2017) and (Fortuna and Nunes, 2018), the paper by (David- son et al., 2017) which presented the Hate Speech Detection dataset used in (Malmasi and Zampieri, 2017) and a few other recent papers such as (ElSh- erief et al., 2018; Gamb¨ack and Sikdar, 2017; Zhang et al., 2018; Badjatiya et al., 2017). A proposal of typology of abusive language sub-tasks is presented in (Waseem et al., 2017). For studies on languages other than English see (Su et al., 2017) on Chinese and (Fiˇser et al., 2017) on Slovene. Finally, for recent discussion on identifying profanity vs. hate speech see (Mal- masi and Zampieri, 2018). This work highlighted the challenges of distinguishing between profan- ity, and threatening language which may not actu- ally contain profane language. Some of the similar and related previous work- shops are Text Analytics for Cybersecurity and …

Content truncated to 500 words