ML_LTU at SemEval-2022 Task 4: T5 Towards Identifying Patronizing and Condescending Language

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 473 - 478 July 14-15, 2022 ©2022 Association for Computational Linguistics ML_LTU at SemEval-2022 Task 4: T5 Towards Identifying Patronizing and Condescending Language Tosin Adewumi, Lama Alkhaled, Hamam Mokayed, Foteini Liwicki and Marcus Liwicki Machine Learning Group EISLAB, SRT Luleå University of Technology firstname.lastname@ltu.se Abstract This paper describes the system used by the Machine Learning Group of LTU in subtask 1 of the SemEval-2022 Task 4: Patronizing and Condescending Language (PCL) Detection. Our system consists of finetuning a pretrained Text-to-Text-Transfer Transformer (T5) and in- novatively reducing its out-of-class predictions. The main contributions of this paper are 1) the description of the implementation details of the T5 model we used, 2) analysis of the successes & struggles of the model in this task, and 3) ablation studies beyond the official submission to ascertain the relative importance of data split. Our model achieves an F1 score of 0.5452 on the official test set. Pérez-Almendros et al. (2020) introduced the dataset for the SemEval-2022 Task 4 (Pérez- Almendros et al., 2022)1. The dataset covers the English language. It is meant to support Natural Language Processing (NLP) models in identifying PCL towards vulnerable communities, such as poor families and refugees. The dataset is designed for 2 subtasks in the competition. Subtask 1 is a bi- nary classification task of predicting the presence of PCL while subtask 2 is a multi-label classifica- tion task of predicting PCL categories. We address subtask 1 in this system paper. PCL is an expression that depicts someone in a compassionate way or shows a superior attitude of the speaker (Pérez-Almendros et al., 2022). PCL identification is important because PCL has been shown to have harmful effects on vulnerable groups (Fox and Giles, 1996; Morris, 2007; Bell, 2013; Wang and Potts, 2019). This task of identifying and categorizing PCL is apparently more challenging than some other types of harmful language because it is subtle and generally used with good intentions (Wang and Potts, 2019; Gilda et al., 2022). The main strategy of our system, to address the challenge, was to use a recent SoTA model (T5) in 1semeval.github.io/SemEval2022/tasks a simple, novel way to reduce out-of-class predic- tions. We discovered that our system achieves a relatively good performance on the task and PCL identification is a challenging task, due to its sub- tle nature. It achieved an F1 score of 0.5452 on the test set while the best score was 0.651. This made us rank 27 (66th percentile) out of 78 and we surpass the official RoBERTa baseline. We per- form error analysis and ablation studies to evaluate the strengths and weaknesses of the model. We contribute the model checkpoint publicly on the HuggingFace hub2 and the T5 code 3 The rest of this paper is organized as follows. Section 1 gives a brief background of related work in PCL. Section 2 gives the system overview of what we used for the task. Section 3 describes the experimental setup for …

Content truncated to 500 words