HITSZ-HLT at SemEval-2021 Task 5: Ensemble Sequence Labeling and Span Boundary Detection for Toxic Span Detection

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 521–526 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 521 HITSZ-HLT at SemEval-2021 Task 5: Ensemble Sequence Labeling and Span Boundary Detection for Toxic Span Detection Qinglin Zhu1,†, Zijie Lin1,†, Yice Zhang1,†, Jingyi Sun1, Xiang Li1 Qihui Lin1, Yixue Dang2, Ruifeng Xu1,‡ 1Joint Lab of HITSZ-CMS, Harbin Institute of Technology(Shenzhen), China 2China Merchants Securities Co., Ltd {zhuqinglin,20S051050,xiangli,1171000607}@stu.hit.edu.cn {lzjjeffery, zhangyc hit}@163.com, dangyixue@cmschina.com.cn xuruifeng@hit.edu.cn Abstract This paper presents the winning system that participated in SemEval-2021 Task 5: Toxic Spans Detection. This task aims to locate those spans that attribute to the text’s toxic- ity within a text, which is crucial for semi- automated moderation in online discussions. We formalize this task as the Sequence Label- ing (SL) problem and the Span Boundary De- tection (SBD) problem separately and employ three state-of-the-art models. Next, we inte- grate predictions of these models to produce a more credible and complement result. Our system achieves a char-level score of 70.83%, ranking 1/91. In addition, we also explore the lexicon-based method, which is strongly inter- pretable and flexible in practice. 1 Introduction 41% of American adults in 2020 are reported ex- periencing some form of harassment1. Increasing incidents of online harassment and cyber violence have spurred researchers to investigate the prob- lem of identifying and filtering offensive speech on the Internet. Most previously published insult detection tasks (Davidson et al., 2017; Xu et al., 2012) and methods (Aroyehun and Gelbukh, 2018; Modha et al., 2018) classify an entire comment (or document) to discern whether the comment is offen- sive or not, but cannot identify specific pieces of the toxic comment. Unlike previous studies, SemEval- 2021 Task5: Toxic Span Detection(Pavlopoulos et al., 2021) requires the identification of the spe- cific toxic spans, which is more innovative and challenging, and a key step towards a successful semi-automatic review of comments. †Authors equally contributed to this work. ‡Corresponding Author: xuruifeng@hit.edu.cn 1https://www.pewresearch.org/internet/2021/01/13/the- state-of-online-harassment/ More formally, toxic span detection is an extrac- tion task, which is usually formalized as a Sequen- tial Labeling (SL) problem, as shown in Figure 1(a), locating those spans by BIO tags. However, SL methods suffer from a huge search space due to the compositionality of labels (the power set of all sentence words), which has been proven in (Lee et al., 2016; Hu et al., 2019a). Therefore, in ad- dition to SL formalization, we also formalize the task as a Span Boundary Detection (SBD) problem, as shown in Figure 1(b), locating those spans by start and end positions. Notice that, when there are multiple spans in a sentence, the matching of start and end positions may be ambiguous during decoding. This shows that theoretically, the SBD formalization is not consistently superior to the SL formalization. Hence, we choose to combine predictions of these two kinds of formalization to produce a more credible and complement result. Our system achieves a char-level score of 70.83%, ranking 1/91. Besides, we also explore the lexicon-based meth- ods, …

Content truncated to 500 words