Unsupervised Embeddings with Graph Auto-Encoders for Multi-domain and Multilingual Hate Speech Detection

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 2196–2204 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 2196 Unsupervised Embeddings with Graph Auto-Encoders for Multi-domain and Multilingual Hate Speech Detection Gretel Liz De la Pe˜na Sarrac´en,1,2 Paolo Rosso1 1Universitat Polit`ecnica de Val`encia, 2Symanto Research gredela@posgrado.upv.es, prosso@dsic.upv.es Abstract Hate speech detection is a prominent and challenging task, since hate messages are often expressed in subtle ways and with characteristics that may vary depending on the author. Hence, many models suffer from the generalization problem. However, retrieving and monitoring hateful content on social media is a current necessity. In this paper, we propose an unsupervised approach using Graph Auto-Encoders (GAE), which allows us to avoid using labeled data when training the representation of the texts. Specifically, we represent texts as nodes of a graph, and use a transformer layer together with a convolutional layer to encode these nodes in a low-dimensional space. As a result, we obtain embeddings that can be decoded into a reconstruction of the original network. Our main idea is to learn a model with a set of texts without supervision, in order to generate embeddings for the nodes: nodes with the same label should be close in the embedding space, which, in turn, should allow us to distinguish among classes. We employ this strategy to detect hate speech in multi-domain and multilingual sets of texts, where our method shows competitive results on small datasets. Keywords: Hate Speech Detection, Unsupervised Embeddings, Graph Auto-Encoders 1. Introduction In this paper we investigate an unsupervised, graph- based approach to learn embeddings for hate speech detection. According to (Fortuna and Nunes, 2018), hate speech can be defined as a language that attacks, diminishes or incites violence against groups based on specific characteristics. Accordingly, the aim of hate speech detection is to discriminate texts that contain hate from those that do not. This is a widely studied task that involves a number of challenges (Poletto et al., 2020). Specifically, we study the problem of data- poor settings that appears in low-resource domains and languages – i.e., in those settings where supervised ap- proach are not able to generalise well. For this, we make use of Graph neural networks (GNNs). They are a framework based on deep learning to operate on graphs. They follow a recursive neigh- borhood aggregation scheme, called message pass- ing, where each node aggregates feature vectors of its neighbors to compute its new feature vector (Xu et al., 2018). After a number of iterations, a node is represented by its transformed feature vector, which captures the structural information within its neighbor- hood. Therefore, GNNs have been effective at tasks thought to have rich relational structure since they can preserve global structure information of a graph in em- beddings. (Wu et al., 2021) provide an overview of recent studies on deep learning approaches for graphs and discuss the applications of GNNs in several areas. (Zhou et al., 2020) present some GNN-based methods …

Content truncated to 500 words