ASRtrans at SemEval-2022 Task 4: Ensemble of Tuned Transformer-based Models for PCL Detection

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 344 - 351 July 14-15, 2022 ©2022 Association for Computational Linguistics ASRtrans at SemEval-2022 Task 4: Ensemble of Tuned Transformer-based Models for PCL Detection Ailneni Rakshitha Rao Indian Institute of Technology, Gandhinagar rao_ailneni@alumni.iitgn.ac.in Abstract Patronizing behavior is a subtle form of bully- ing and when directed towards vulnerable com- munities, it can arise inequalities. This paper describes our system for Task 4 of SemEval- 2022: Patronizing and Condescending Lan- guage Detection (PCL). We participated in both the sub-tasks and conducted extensive experi- ments to analyze the effects of data augmenta- tion and loss functions used, to tackle the prob- lem of class imbalance. We explore whether large transformer-based models can capture the intricacies associated with PCL detection. Our solution consists of an ensemble of the RoBERTa model which is further trained on ex- ternal data and other language models such as XLNeT, Ernie-2.0, and BERT. We also present the results of several problem transformation techniques such as Classifier Chains, Label Powerset, and Binary relevance for multi-label classification. 1 Introduction In this paper, we discuss various linguistic tech- niques used for detecting Patronizing and conde- scending language (PCL). This task particularly poses a new challenge in the field of NLP because of its subjectivity, subtle usage, and requirement of world knowledge. A person is said to be con- descending or patronizing when he/she uses a su- perior tone to talk down to people or tries to raise pity by describing their situation. Even though people often use PCL with good intentions, it en- courages stereotyping, discrimination and leads to greater exclusion. Therefore, it is important to de- vise methods that facilitate the automatic detection of PCL. SemEval 2022 Task 4 (Pérez-Almendros et al., 2022) is the first attempt to detect the usage of PCL towards vulnerable communities. It has two subtasks: In sub-task A, we need to detect if the given paragraph contains PCL or not. Sub-task B aims to classify PCL text further among potential intersecting categories. The two main challenges faced in this task are extreme class imbalance in the dataset and the sub- tle nature of PCL present in the text which makes it hard, even for humans to classify it correctly. Also, the model needs to differentiate between actual news of extremely vulnerable situations from text containing PCL. Pre-trained language models such as BERT, XL- NeT, etc., have emerged as the state-of-the-art mod- els for many NLP tasks such as text classifica- tion, machine translation, sequence tagging, etc. However, they are trained on typical day-to-day texts. PCL text is not trivial and classifying certain classes requires some level of world knowledge and commonsense reasoning. In this paper we conduct detailed experiments using the following models: RoBERTa (Liu et al., 2019), XLNet (Yang et al., 2020), Ernie-2.0 (Sun et al., 2019), BERT- large (Devlin et al., 2019), label specific attention network (LSAN) (Xiao et al., 2019) and their en- sembles for PCL detection. Additionally, we test the …

Content truncated to 500 words