SATLab at SemEval-2022 Task 4: Trying to Detect Patronizing and Condescending Language with only Character and Word N-grams

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 490 - 495 July 14-15, 2022 ©2022 Association for Computational Linguistics SATLab at SemEval-2022 Task 4: Trying to Detect Patronizing and Condescending Language with only Character and Word N-grams Yves Bestgen Statistical Analysis of Text Laboratory (SATLab) Université catholique de Louvain Place Cardinal Mercier, 10 1348 Louvain-la-Neuve, Belgium yves.bestgen@uclouvain.be Abstract A logistic regression model only fed with char- acter and word n-grams is proposed for the SemEval-2022 Task 4 on Patronizing and Con- descending Language Detection (PCL). It ob- tained an average level of performance, well above the performance of a system that tries to guess without using any knowledge about the task, but much lower than the best teams. To facilitate the interpretation of the performance scores, the F1 measure, the best level of perfor- mance of a system that tries to guess without using any knowledge is calculated and used to correct the F1 scores in the manner of a Kappa. As the proposed model is very similar to the one that performed well on a task requiring to automatically identify hate speech and offen- sive content, this paper confirms the difficulty of PCL detection. 1 Introduction This paper presents the SATLab’s participation in SemEval-2022 Task 4: Patronizing and Conde- scending Language Detection. It is a very recent task that aims at identifying passages in texts in which a person is condescending to a vulnerable community (Pérez-Almendros et al., 2020; Wang and Potts, 2019). The task organizers have pro- posed to identify this type of discourse in para- graphs extracted from news stories published in English-speaking media. The Patronizing and Condescending Language (PCL) Detection task is part of the large family of tasks that aim at automatically identifying prob- lematic language, whether in media or on social networks. Examples include the detection of hate speech and offensive content, fakenews and hy- perpartisan news (Kiesel et al., 2019). The PCL challenge, however, differs from these cases in one important characteristic. As pointed out by (Pérez- Almendros et al., 2020), using PCL is not always conscious and the intention of its author is often positive. Nevertheless, PCL tends to indirectly de- mean the community in question and reinforce neg- ative stereotypes about it. It is therefore important to develop procedures to identify it. Recently, the SATLab performed well on a task requiring to automatically identify hate speech and offensive content in tweets using a classical super- vised algorithm only fed with character n-grams (Bestgen, 2021b). It was especially effective for resource-poor languages (e.g., Marathi). For En- glish, its performance (F1 = 0.782) was aver- age, but nevertheless relatively close to the best team (F1 = 0.831) that used complementary re- sources, pre-trained embeddings and deep learning approaches. These performances suggest that the proposed approach could be a interesting baseline to evaluate the benefits of these more complex ap- proaches. The goal of SATLab’s participation in the PCL detection task was to evaluate whether this find- ing can be generalized …

Content truncated to 500 words