SOLID: A Large-Scale Semi-Supervised Dataset for Offensive Language Identification

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 915–928 August 1–6, 2021. ©2021 Association for Computational Linguistics 915 SOLID: A Large-Scale Semi-Supervised Dataset for Offensive Language Identification Sara Rosenthal1, Pepa Atanasova2, Georgi Karadzhov3, Marcos Zampieri4, Preslav Nakov5 1IBM Research, USA, 2University of Copenhagen, Denmark, 3University of Cambridge, UK, 4Rochester Institute of Technology, USA, 5Qatar Computing Research Institute, HBKU, Qatar sjrosenthal@us.ibm.com Abstract The widespread use of offensive content in so- cial media has led to an abundance of research in detecting language such as hate speech, cyberbullying, and cyber-aggression. Recent work presented the OLID dataset, which fol- lows a taxonomy for offensive language identi- fication that provides meaningful information for understanding the type and the target of of- fensive messages. However, it is limited in size and it might be biased towards offensive language as it was collected using keywords. In this work, we present SOLID, an expanded dataset, where the tweets were collected in a more principled manner. SOLID contains over nine million English tweets labeled in a semi- supervised fashion. We demonstrate that using SOLID along with OLID yields sizable perfor- mance gains on the OLID test set for two dif- ferent models, especially for the lower levels of the taxonomy. 1 Introduction Offensive language in social media has become a concern for governments, online communities, and social media platforms. Free speech is an im- portant right, but moderation is needed in order to avoid unexpected serious repercussions. In fact, this is so serious that many countries have passed or are planning legislation that makes platforms responsible for their content, e.g., the Online Harm Bill (HM Government, 2019) in the UK and the Digital Services Act (European Commission, 2020) in the EU. Even in the United States, content moder- ation or the lack thereof can have significant impact on business (e.g., Parler was denied server space), government (U.S. Capitol Riots), and individuals (hate speech is linked to self-harm). Explainabil- ity is needed to indicate in detail why content has WARNING: This paper contains tweet examples and words that are offensive in nature. been deleted or flagged as inappropriate. Moreover, users can be educated by such feedback to avoid future biases. There have been several areas of work in the detection of offensive language (Basile et al., 2019; Fortuna and Nunes, 2018; Ranasinghe and Zampieri, 2020), covering overlapping character- istics such as toxicity, hate speech, cyberbullying, and cyber-aggression. Further, using a hierarchi- cal approach to analyze different aspects of the offensive content, such as the type and the target of the offense, helps provide explainability. The Of- fensive Language Identification Dataset, or OLID, (Zampieri et al., 2019a) is one such example, and it has been widely used in research. OLID con- tains 14,100 English tweets, which were manually annotated using a three-level taxonomy: A: Offensive Language Detection B: Categorization of Offensive Language C: Offensive Language Target Identification The taxonomy proposed in OLID makes it possi- ble to represent different kinds of offensive con- tent as a function of …

Content truncated to 500 words