CRASS: A Novel Data Set and Benchmark to Test Counterfactual Reasoning of Large Language Models

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 202–210 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 202 Identification of Multiword Expressions in Tweets for Hate Speech Detection Nicolas Zampieri1, Carlos Ramisch2, Irina Illina1, Dominique Fohr1 1Lorraine University, CNRS, Inria, Loria, F-54000 Nancy, France 2Aix-Marseille University, Universit´e de Toulon, CNRS, LIS, Marseille, France nicolas.zampieri@inria.fr, {illina, fohr}@loria.fr, carlos.ramisch@lis-lab.fr Abstract Multiword expression (MWE) identification in tweets is a complex task due to the complex linguistic nature of MWEs combined with the non-standard language use in social networks. MWE features were shown to be helpful for hate speech detection (HSD). In this article, we present joint experiments on these two related tasks on English Twitter data: first we focus on the MWE identification task, and then we observe the influence of MWE-based features on the HSD task. For MWE identification, we compare the performance of two systems: lexicon-based and deep neural networks-based (DNN). We experimentally evaluate seven configurations of a state-of-the-art DNN system based on recurrent networks using pre-trained contextual embeddings from BERT. The DNN-based system outperforms the lexicon-based one thanks to its superior generalisation power, yielding much better recall. For the HSD task, we propose a new DNN architecture for incorporating MWE features. We confirm that MWE features are helpful for the HSD task. Moreover, the proposed DNN architecture beats previous MWE-based HSD systems by 0.4 to 1.1 F-measure points on average on four Twitter HSD corpora. Keywords: multiword expressions, hate speech, social media 1. Introduction A multiword expression (MWE) is a lexicalised com- bination of two or more lexemes which exhibits some form of idiomaticity (Baldwin and Kim, 2010). Auto- matic identification of MWEs is a difficult task in natural language processing because, among others, MWEs can have discontinuities and overlaps (Constant et al., 2017). Moreover, only a few corpora annotated in terms of MWEs are available. In this article, we study the robustness of MWE identi- fication systems on non-standard texts, namely tweets. Indeed, tweets often employ non-standard syntax and contain spelling mistakes, abbreviations, etc. We hy- pothesise that, under these conditions, the MWE identi- fication task becomes even more difficult. Hate speech is commonly defined as a communication that disparages a person or a group based on some char- acteristic such as race, colour, gender, etc. (Nockeby, 2000). Manual moderation of harmful tweets is not possible due to the huge number of tweets posted every day. Thus, automatic methods to support social media moderation can potentially help fight online harassment, cancellation, polarisation, misinformation, etc. In this work, we are interested in studying the impact of different MWE identification systems for automatic hate speech detection (HSD). Previously, Stankovi´c et al. (2020) and Zampieri et al. (2021) have shown that MWEs are helpful for this task. We compare two auto- matic MWE identification systems: the first one utilises a look-up method on a lexicon, the second one is based on a deep neural network (DNN). The identified MWEs are employed as …

Content truncated to 500 words