Xu at SemEval-2022 Task 4: Pre-BERT Neural Network Methods vs Post-BERT RoBERTa Approach for Patronizing and Condescending Language Detection

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 479 - 484 July 14-15, 2022 ©2022 Association for Computational Linguistics Xu at SemEval-2022 Task 4: Pre-BERT Neural Network Methods vs Post-BERT RoBERTa Approach for Patronizing and Condescending Language Detection Jinghua Xu University of Tübingen jinghua.xu@student.uni-tuebingen.de Abstract This paper describes my participation in the SemEval-2022 Task 4: Patronizing and Conde- scending Language Detection. I participate in both subtasks: Patronizing and Condescending Language (PCL) Identification and Patronizing and Condescending Language Categorization, with the main focus put on subtask 1. The ex- periments compare pre-BERT neural network (NN) based systems against post-BERT pre- trained language model RoBERTa. This re- search finds that NN-based systems in the ex- periments perform worse on the task compared to the pretrained language models. The top- performing RoBERTa system is ranked 26 out of 78 teams (F1-score: 54.64) in subtask 1, and 23 out of 49 teams (F1-score: 30.03) in subtask 2. 1 Introduction An entity is considered to engage Patronizing and Condescending Language (PCL) when its language use presents a superior attitude towards others or depicts them in a compassionate way (Pérez- Almendros et al., 2020). Such language is of- ten used toward vulnerable communities such as women, refugees, and homeless people. These unfair treatments of the vulnerable groups are be- lieved to result in further exclusion and inequalities in society. Compared to other types of harmful language (e.g. hate speech), PCL is considered more subtle and unconscious. Given the negative effects of PCL on society and its subtle nature, en- abling computers to identify and categorize PCL presents an interesting technical challenge to the NLP community. This paper describes my participation in both subtasks of the SemEval-2022 Task 4: Patronizing and Condescending Language Detection (Pérez- Almendros et al., 2022). Subtask 1, Patronizing and Condescending Language Identification is a binary text classification task to predict whether a given paragraph contains PCL or not. Subtask 2, Patronizing and Condescending Language Cat- egorization is a multi-label classification task to identify the categories of a given paragraph accord- ing to the taxonomy defined in Pérez-Almendros et al. (2020), which categorizes PCL into 7 types: 1) Unbalanced power relations 2) Shallow solution 3) Presupposition 4) Authority voice 5) Metaphor 6) Compassion 7) The poorer, the merrier. The dataset (Pérez-Almendros et al., 2020) contains an- notated paragraphs in English, collected from news stories in 20 English-speaking countries.1 The focus of my experiments is primarily on subtask 1, meanwhile, this paper also proposes a solution to subtask 2. For subtask 1, the exper- iments compare pre-BERT neural network (NN) based systems including a majority voting system of NN models against pretrained language model RoBERTa. The experiments start with building in- dividual NN models from the most basic artificial neural network (ANN) to long short-term memory network (LSTM) models following previous work on NN for text classification. It was found that the NN-based systems in the experiments perform worse on this task in comparison to the pretrained language models. The best-performing NN-based …

Content truncated to 500 words