Cisco at SemEval-2021 Task 5: What’s Toxic?: Leveraging Transformers for Multiple Toxic Span Extraction from Online Comments

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 249–257 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 249 Cisco at SemEval-2021 Task 5: What’s Toxic?: Leveraging Transformers for Multiple Toxic Span Extraction from Online Comments Sreyan Ghosh Cisco Systems, Bangalore, India MIDAS Lab, IIIT-Delhi, India sreyghos@cisco.com Sonal Kumar Cisco Systems, Bangalore, India sonalkum@cisco.com Abstract Social network platforms are generally used to share positive, constructive, and insightful content. However, in recent times, people of- ten get exposed to objectionable content like threat, identity attacks, hate speech, insults, ob- scene texts, offensive remarks or bullying. Ex- isting work on toxic speech detection focuses on binary classification or on differentiating toxic speech among a small set of categories. This paper describes the system proposed by team Cisco for SemEval-2021 Task 5: Toxic Spans Detection, the first shared task focusing on detecting the spans in the text that attribute to its toxicity, in English language. We ap- proach this problem primarily in two ways: a sequence tagging approach and a dependency parsing approach. In our sequence tagging ap- proach we tag each token in a sentence under a particular tagging scheme. Our best perform- ing architecture in this approach also proved to be our best performing architecture over- all with an F1 score of 0.6922, thereby plac- ing us 7th on the final evaluation phase leader- board. We also explore a dependency parsing approach where we extract spans from the in- put sentence under the supervision of target span boundaries and rank our spans using a biaffine model. Finally, we also provide a de- tailed analysis of our results and model perfor- mance in our paper. 1 Introduction It only takes one toxic comment to sour an online discussion. The threat of abuse and harassment online leads many people to stop expressing them- selves and give up on seeking different opinions. Toxic content is ubiquitous in social media plat- forms like Twitter, Facebook, Reddit, the increase of which is a major cultural threat and has already lead to a crime against minorities (Williams et al., 2020). Toxic text in online social media varies de- pending on targeted groups (e.g. women, LGBT, gay, African, immigrants) or the context (e.g. pro- trump discussion or the metoo movement). Toxic Text online has often been broadly classified by researchers into different categories like hate, of- fense, hostility, aggression, identity attacks, and cyberbullying. Though the use of various terms for equivalent tasks makes them incomparable at times (Fortuna et al., 2020), toxic speech or spans in this particular task, SemEval-2021 Task 5 (Pavlopoulos et al., 2021), has been considered as a super-set of all the above sub-types. Figure 1: Toxic spans in sentences While a lot of models have claimed to achieve state-of-the-art results on various datasets, it has been observed that most models fail to generalize (Arango et al., 2019; Gr¨ondahl et al., 2018). The models tend to classify comments as toxic that have a reference to certain commonly-attacked entities (e.g. gay, …

Content truncated to 500 words