Team Stanford ACMLab at SemEval 2022 Task 4: Textual Analysis of PCL Using Contextual Word Embeddings

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 418 - 420 July 14-15, 2022 ©2022 Association for Computational Linguistics Team Stanford ACMLab at SemEval-2022 Task 4: Textual Analysis of PCL Using Contextual Word Embeddings Upamanyu Dass-Vattam, Spencer Wallace, Rohan Sikand, Zach Witzel, Jillian Tang Stanford University {udvattam, spenwall, rsikand, zachwitz, jiltang}@stanford.edu Abstract We propose the use of a contextual embedding based-neural model on strictly textual inputs to detect the presence of patronizing or conde- scending language (PCL). We finetuned a pre- trained BERT model to detect whether or not a paragraph contained PCL (Subtask 1), and fur- thermore finetuned another pre-trained BERT model to identify the linguistic techniques used to convey the PCL (Subtask 2). Results show that this approach is viable for binary classi- fication of PCL, but breaks when attempting to identify the PCL techniques. Our system placed 32/79 for subtask 1, and 40/49 for sub- task 2. 1 Introduction The goal of the task is to be able to identify whether or not a piece of text contains condescending or patronizing language, and if it contains patronizing language identity which linguistic techniques are used to express that sentiment (P’erez-Almendros et al., 2022). Studies have shown that PCL fuels discriminatory behavior, creates and feeds stereo- types, subtly blames needy individuals for their problems, and oversimplifies problems. In general, PCL makes it harder for needy communities to get the help they need and reach total inclusivity (P’erez-Almendros et al., 2020). This is obviously negative, and being able to combat it with AI may help automate the process and get past inherent biases that humans identifying PCL may have. Our system’s goal is to use contextualized word embeddings to feed the model, and thus have the model analyze the semantics of the text. We specif- ically focused on a purely textual analysis and did not provide the model with any metadata to see if the model could learn the PCL patterns just from the text, because although real world usage would likely include those features, the ability to learn from the text itself would be useful and more uni- versally applicable. In this task, we learnt that it is easier for a model to simply detect the presence of PCL than the tech- niques used in a piece of PCL. This is likely be- cause of both unequal distributions of data and the fact that the context for the types of PCL likely look very similar, making the model default to a category of PCL with a higher frequency when not incredi- bly clear. We ranked 32/70 on subtask 1, which is detecting the presence of PCL, and ranked 40/49 on subtask 2, which involves identifying the PCL tech- niques used. In particular, the model struggled with the ‘authority voice’ and ‘metaphor’ categories. 2 Task Overview The dataset provided was the Don’t Patronize Me! Dataset (P’erez-Almendros et al., 2020), which is a collection of paragraphs mentioning vulnerable communities and published in media in 20 English speaking countries. …

Content truncated to 500 words