BEIKE NLP at SemEval-2022 Task 4: Prompt-Based Paragraph Classification for Patronizing and Condescending Language Detection

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 319 - 323 July 14-15, 2022 ©2022 Association for Computational Linguistics BEIKE NLP at SemEval-2022 Task 4: Prompt-Based Paragraph Classification for Patronizing and Condescending Language Detection Yong Deng, Chenxiao Dou∗, Liangyu Chen Deqiang Miao, Xianghui Sun, Baochang Ma, Xiangang Li AI Center@Beike, China {dengyong013, douchenxiao001,chenliangyu003}@ke.com Abstract PCL detection task is aimed at identifying and categorizing language that is patronizing or con- descending towards vulnerable communities in the general media. Compared to other NLP tasks of paragraph classification, the negative language presented in the PCL detection task is usually more implicit and subtle to be recog- nized, making the performance of common text- classification approaches disappointed. Target- ing the PCL detection problem in SemEval- 2022 Task 4, in this paper, we give an intro- duction to our team’s solution, which exploits the power of prompt-based learning on para- graph classification. We reformulate the task as an appropriate cloze prompt and use pre- trained Masked Language Models to fill the cloze slot. For the two subtasks, binary classifi- cation and multi-label classification, DeBERTa model is adopted and fine-tuned to predict masked label words of task-specific prompts. On the evaluation dataset, for binary classifi- cation, our approach achieves an F1-score of 0.6406; for multi-label classification, our ap- proach achieves an macro-F1-score of 0.4689 and ranks first in the leaderboard. 1 Introduction Patronizing and Condescending Language (PCL) towards vulnerable communities in the media has become a hot issue, which is heatedly discussed in society at present. The language often refers to the discourse which is published with a mixture of pity and superiority towards unprivileged groups. Such attitude is always adopted unconsciously or even out of kindness, however, its negative effect indeed exacerbates social prejudice and routinizes discrimination towards disadvantaged people (Ng, 2007). Despite the PCL emerging in numerous me- dia, its style can not be precisely captured by far, since the usage of PCL is commonly unintended, ∗ Corresponding Author unaffected, and subjective. This poses many chal- lenges to the detection of PCL, which attracts great attention from the NLP community. Even though there exist substantial NLP studies on paragraph identification in various fields, research specifically for PCL identification has not been yet seriously introduced (Pérez-Almendros et al., 2020). Nowa- days with social discrimination continuously ris- ing, an effective approach, which can automatically identify PCL towards vulnerable communities, be- comes more and more necessary and important to our society. To encourage more research on the PCL prob- lem, SemEval-2022 Task 4 (Pérez-Almendros et al., 2022) provides an English PCL dataset for language-modeling study and evaluation. The main task contains two PCL-classification subtasks, one for binary classification (SubTask 1) and the other for multi-label classification (SubTask 2). Given a paragraph, SubTask 1 is aimed to identify whether or not it is written with PCL style. F1 over the positive class is taken as the evaluation metric of this task. For the same paragraph, the goal of Sub- Task 2 is to classify which …

Content truncated to 500 words