K-MHaS: A Multi-label Hate Speech Detection Dataset in Korean Online News Comment

Proceedings of the 29th International Conference on Computational Linguistics, pages 3530–3538 October 12–17, 2022. 3530 K-MHaS: A Multi-label Hate Speech Detection Dataset in Korean Online News Comment Jean Lee1 Taejun Lim1 Heejun Lee3 Bogeun Jo3 Yangsok Kim4 Heegeun Yoon5 Soyeon Caren Han1,2∗ The University of Sydney1 The University of Western Australia2 BigWave AI3 Keimyung University4 National Information Society Agency5 {jean.lee, tlim6535, caren.han}@sydney.edu.au {leej3471, iambgjo}@gmail.com yangsok.kim@kmu.ac.kr hgyoon@nia.or.kr Abstract Online hate speech detection has become an important issue due to the growth of online content, but resources in languages other than English are extremely limited. We introduce K-MHaS1, a new multi-label dataset for hate speech detection that effectively handles Ko- rean language patterns. The dataset consists of 109k utterances from news comments and pro- vides a multi-label classification using 1 to 4 la- bels, and handles subjectivity and intersection- ality. We evaluate strong baselines on K-MHaS. KR-BERT with a sub-character tokenizer out- performs others, recognizing decomposed char- acters in each hate speech class. 1 Introduction The growth of online content including social media (Zampieri et al., 2020), news comments (Gao and Huang, 2017), Wikipedia (Wulczyn et al., 2017), and in-game chat (Weld et al., 2021) presents challenges in detecting hate speech us- ing advanced Natural Language Processing. Hate speech is language that attacks or diminishes indi- viduals or groups based on certain characteristics such as physical appearance, religion, gender, or other attributes, and it can occur across different linguistic styles (Fortuna and Nunes, 2018). Hate speech detection is intrinsically a complex task (Wang et al., 2020) due to the fuzzy boundary with other overlapping concepts such as abusive lan- guage (Nobata et al., 2016), toxic comments (Wul- czyn et al., 2017), or offensive language (Davidson et al., 2017). Recently, the rise in popularity of Korean TV, movies, and music (e.g. Squid Game, BTS) has led to many young people showing an interest in learning Korean. This phenomenon could result in exposure to harmful content and hate speech in ∗Corresponding author (caren.han@sydney.edu.au) 1The dataset is available at https://github.com/adlnlp/K- MHaS. Korean. However, (1) the most common language in hate speech research is English and only limited resources are available in other languages such as Arabic (Mubarak et al., 2017), Dutch (Caselli et al., 2021), and Korean (Moon et al., 2020). In addition, most datasets are annotated (2) using a single label classification of particular aspects, even though the subjectivity of hate speech cannot be explained with a mutually exclusive annotation scheme. We propose K-MHaS, a Korean multi-label hate speech detection dataset that allows overlapping labels associated with intersectionality, a concept from sociology that identifies combined attributes (Crenshaw, 1989). Our dataset consists of 109,692 utterances from Korean online news comments, labeled with 8 fine-grained hate speech classes. K- MHaS is compatible with previous work on hate speech in other languages, by providing binary clas- sification and multi-label classification from 1(one) to 4(four) labels. We investigate the K-MHaS dataset by analyz- ing label distribution, keywords, and label pairs. In addition, we provide strong baseline pre-trained language …

Content truncated to 500 words