PALI-NLP at SemEval-2022 Task 4: Discriminative Fine-tuning of Transformers for Patronizing and Condescending Language Detection

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 335 - 343 July 14-15, 2022 ©2022 Association for Computational Linguistics PALI-NLP at SemEval-2022 Task 4: Discriminative Fine-tuning of Transformers for Patronizing and Condescending Language Detection Dou Hu and Mengyuan Zhou and Xiyang Du and Mengfei Yuan and Meizhi Jin and Lianxin Jiang and Yang Mo and Xiaofeng Shi Ping An Life Insurance Company of China, Ltd. {HUDOU470, ZHOUMENGYUAN425, DUXIYANG037, YUANMENGFEI854, JINMEIZHI005, JIANGLIANXIN769, MOYANG853, SHIXIAOFENG309} @pingan.com.cn Abstract Patronizing and condescending language (PCL) has a large harmful impact and is difficult to de- tect, both for human judges and existing NLP systems. At SemEval-2022 Task 4, we pro- pose a novel Transformer-based model and its ensembles to accurately understand such lan- guage context for PCL detection. To facilitate comprehension of the subtle and subjective na- ture of PCL, two fine-tuning strategies are ap- plied to capture discriminative features from di- verse linguistic behaviour and categorical distri- bution. The system achieves remarkable results on the official ranking, including 1st in Subtask 1 and 5th in Subtask 2. Extensive experiments on the task demonstrate the effectiveness of our system and its strategies. 1 Introduction “Don’t worry, I know this is a mistake you usually make, we all make it sometimes, but I am bring- ing you a solution.”, which is a typical example of Patronizing and Condescending Language (PCL) (Giles et al., 1993; Huckin, 2002), shows a supe- rior attitude and apparent kindness towards others, while is generally expressed unconsciously. The impact of PCL can potentially be very harmful, as it feeds inequalities and routinizes discrimination (Ng, 2007), especially if it is geared towards vul- nerable communities in the media. If we are able to detect and identify when we are condescending or patronizing towards others, a corrective action (e.g., a more inclusive message) could be taken for a more responsible communication. Recently, some works (Wang and Potts, 2019; Sap et al., 2020) on PCL are gradually emerging in NLP community. Remarkably, Pérez-Almendros et al. (2020) have shown that general pre-trained language models (Devlin et al., 2019; Liu et al., 2019) can achieve nontrivial performance. How- ever, the behaviour of PCL is usually more uncon- scious, subtler, and subjective than other harmful types of discourse that are widely studied, i.e., hate speech (Basile et al., 2019), offensive language (Zampieri et al., 2019), intended sarcasm (Du et al., 2022), fake news (Zhang et al., 2021b) and ru- mor (Wei et al., 2021). These characteristics make PCL detection a difficult challenge, both for human judges and existing NLP systems. To address this, we propose a novel Transformer- based model BERT-PCL (and its ensembles) with two discriminative fine-tuning strategies, to accu- rately understand such language context for PCL detection. The two strategies are grouped layer- wise learning rate decay (Grouped LLRD) and weighted random sampler (WRS), and both are beneficial for task-adaptive fine-tuning based on language models. A brief description of these two strategies is as follows: 1) As different layers capture differ- ent types …

Content truncated to 500 words