NLRG at SemEval-2021 Task 5: Toxic Spans Detection Leveraging BERT-based Token Classification and Span Prediction Techniques

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 233–242 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 233 NLRG at SemEval-2021 Task 5: Toxic Spans Detection Leveraging BERT-based Token Classification and Span Prediction Techniques Gunjan Chhablani∗ Dept. of CS&IS BITS Pilani, Goa, India chhablani.gunjan@gmail.com Abheesht Sharma∗ Dept. of CS&IS BITS Pilani, Goa, India f20171014@goa.bits-pilani.ac.in Harshit Pandey∗ Dept. of CS Pune University, India hp2pandey1@gmail.com Yash Bhartia Dept. of CS&IS BITS Pilani, Goa, India f20190151@goa.bits-pilani.ac.in Shan Suthaharan Dept. of CS UNC-Greensboro, NC, USA s suthah@uncg.edu Abstract Toxicity detection of text has been a popular NLP task in the recent years. In SemEval- 2021 Task-5 Toxic Spans Detection, the fo- cus is on detecting toxic spans within En- glish passages. Most state-of-the-art span de- tection approaches employ various techniques, each of which can be broadly classified into Token Classification or Span Prediction ap- proaches. In our paper, we explore simple versions of both of these approaches and their performance on the task. Specifically, we use BERT-based models - BERT, RoBERTa, and SpanBERT for both approaches. We also com- bine these approaches and modify them to bring improvements for Toxic Spans predic- tion. To this end, we investigate results on four hybrid approaches - Multi-Span, Span+Token, LSTM-CRF, and a combination of predicted offsets using union/intersection. Additionally, we perform a thorough ablative analysis and analyze our observed results. Our best submis- sion - a combination of SpanBERT Span Pre- dictor and RoBERTa Token Classifier predic- tions - achieves an F1 score of 0.6753 on the test set. Our best post-eval F1 score is 0.6895 on intersection of predicted offsets from top- 3 RoBERTa Token Classification checkpoints. These approaches improve the performance by 3% on average than those of the shared base- line models - RNNSL and SpaCy NER. 1 Introduction Offensive language can include various categories such as threats, vilification, insults, calumniation, discrimination and swearing (Pavlopoulos et al., 2019). Detection of such language is necessary for ease of moderation of content on social media. De- spite their popularity, toxicity detection tasks have focused majorly on sequence classification, rather ∗ Equal contribution. Author ordering determined by coin flip. than sequence tagging. Finding which spans make a comment or document toxic in nature is crucial in explaining the reasons behind their toxicity. Ad- ditionally, such attributions would allow for more efficient semi-automated quality-based moderation of content, especially for verbose documents, in comparison to quantitative toxicity scores. In SemEval-2021 Task-5, Pavlopoulos et al. (2021) provide a dataset of 10k English texts fil- tered from Civil Comments (Borkan et al., 2019) dataset. Each text is crowd-annotated with char- acter offsets that make the text toxic. The task is to predict these character offsets given the text. The work presented in this paper aims to provide a comprehensive analysis of simple Token Classifica- tion (TC) and Span Prediction (SP) methods across multiple BERT-based models - BERT (Devlin et al., 2019), RoBERTa (Liu et al., 2019) and SpanBERT (Joshi et al., 2020). Additionally, we experiment …

Content truncated to 500 words