PiCkLe at SemEval-2022 Task 4: Boosting Pre-trained Language Models with Task Specific Metadata and Cost Sensitive Learning

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 464 - 472 July 14-15, 2022 ©2022 Association for Computational Linguistics PiCkLe at SemEval-2022 Task 4: Boosting Pre-trained Language Models with Task Specific Metadata and Cost Sensitive Learning Manan Suri Netaji Subhas University of Technology New Delhi, India manan.suri.ug20@nsut.ac.in Abstract This paper describes our system for Task 4 of SemEval 2022: Patronizing and Condescend- ing Language Detection. Patronizing and Con- descending Language (PCL) refers to language used with respect to vulnerable communities that portrays them pitifully and is reflective of a sense of superiority. Task 4 involved bi- nary classification (Subtask 1) and multi-label classification (Subtask 2) of Patronizing and Condescending Language (PCL). For our sys- tem, we experimented with fine-tuning differ- ent transformer-based pre-trained models in- cluding BERT, DistilBERT, RoBERTa and AL- BERT. Further, we have used token separated metadata to improve our model by helping it contextualize different communities with re- spect to PCL. We faced the challenge of class imbalance, which we solved by experimenting with different class weighting schemes. Our models were effective in both subtasks, with the best performance coming out of models with Effective Number of Samples (ENS) class weighting and token separated metadata in both subtasks. For subtask 1 and subtask 2, our best models were finetuned BERT and RoBERTa models respectively. 1 Introduction Patronizing and Condescending Language (PCL) in the context of vulnerable communities refers to language that portrays a sense of superiority or has a tendency to view communities with a pitiful and passionate lens. PCL works subtly and is intricately associated with the way that words and phrases are used. This makes it difficult to classify PCL as com- pared to overtly offensive language where the na- ture of words and phrases is clearly negative. Since the harms associated with PCL are not always evi- dent, it is often used inadvertently by actors trying to help these communities. Recognising PCL is important because the expression of PCL leads to a paradigm where discrimination and injustices are routinised and made less visible (Ng, 2007). Use of PCL also feeds into stereotypes (Fiske, 1993), re- inforces societal power dynamics and avoids deep- rooted societal problems, providing surface-level solutions for the same (Chouliaraki, 2010). Task 4 of SemEval 2022 (Pérez-Almendros et al., 2022) aims to identify PCL with Subtask 1 working to- wards binary classification and Subtask 2 working towards the multi-label classification of PCL. Our strategy involves using state-of-the-art Pre- Trained Language Models (PLMs) and finetuning them for our specific task. We work with BERT (Devlin et al., 2019), RoBERTa (Liu et al., 2019), DistilBERT (Sanh et al., 2019), ALBERT (Lan et al., 2019) as our models. Additionally, we de- sign effective and simple approaches to optimize our models, by experimenting with different cost- sensitive class weighting methods and working with token separated metadata to enhance perfor- mance. With an increasing number of PLMs, each having millions of parameters and being computa- tionally expensive to train, it is essential to make …

Content truncated to 500 words