Transferring Knowledge via Neighborhood-Aware Optimal Transport for Low-Resource Hate Speech Detection

Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 453–467 November 20–23, 2022. ©2022 Association for Computational Linguistics 453 Transferring Knowledge via Neighborhood-Aware Optimal Transport for Low-Resource Hate Speech Detection Tulika Bose Irina Illina Dominique Fohr Universite de Lorraine, CNRS, Inria, LORIA, F-54000 Nancy, France {tulika.bose, illina, dominique.fohr}@loria.fr Abstract Warning: this paper contains content that may be offensive and distressing. The concerning rise of hateful content on on- line platforms has increased the attention to- wards automatic hate speech detection, com- monly formulated as a supervised classification task. State-of-the-art deep learning-based ap- proaches usually require a substantial amount of labeled resources for training. However, an- notating hate speech resources is expensive, time-consuming, and often harmful to the an- notators. This creates a pressing need to trans- fer knowledge from the existing labeled re- sources to low-resource hate speech corpora with the goal of improving system performance. For this, neighborhood-based frameworks have been shown to be effective. However, they have limited flexibility. In our paper, we propose a novel training strategy that allows flexible modeling of the relative proximity of neighbors retrieved from a resource-rich corpus to learn the amount of transfer. In particular, we incor- porate neighborhood information with Optimal Transport, which permits exploiting the geome- try of the data embedding space. By aligning the joint embedding and label distributions of neighbors, we demonstrate substantial improve- ments over strong baselines, in low-resource scenarios, on different publicly available hate speech corpora. 1 Introduction With the alarming spread of Hate Speech (HS) in social media, Natural language Processing tech- niques have been used to develop automatic HS detection systems, typically to aid manual con- tent moderation. Although deep learning-based approaches (Mozafari et al., 2019; Badjatiya et al., 2017) have become state-of-the-art in this task, their performance depends on the size of the la- beled resources available for training (Lee et al., 2018; Alwosheel et al., 2018). Annotating a large corpus for HS is considerably time-consuming, expensive, and harmful to human annotators (Schmidt and Wiegand, 2017; Malmasi and Zampieri, 2018; Poletto et al., 2019; Sarwar et al., 2022). Moreover, models trained on existing labeled HS corpora have shown poor generaliza- tion when evaluated on new HS content (Yin and Zubiaga, 2021; Arango et al., 2019; Swamy et al., 2019; Karan and Šnajder, 2018). This is due to the differences across these corpora, such as sampling strategies (Wiegand et al., 2019), varied topics of discussion (Florio et al., 2020; Saha and Sindhwani, 2012), varied vocabularies, and different victims of hate. Thus, to address these challenges, here we aim to devise a strategy that can effectively trans- fer knowledge from a resource-rich source corpus with a higher amount of annotated content to a low- resource target corpus with fewer labeled instances. One popular way to address this is transfer learn- ing. For instance, Mozafari et al. (2019) fine-tune a large-scale pre-trained language model, BERT …

Content truncated to 500 words