HITMI&T at SemEval-2022 Task 4: Investigating Task-Adaptive Pretraining And Attention Mechanism On PCL Detection

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 438 - 444 July 14-15, 2022 ©2022 Association for Computational Linguistics HITMI&T at SemEval-2022 Task 4: Investigating Task-Adaptive Pretraining And Attention Mechanism On PCL Detection Zihang Liu,Yancheng He,Feiqing Zhuang,Bing Xu∗ Machine Intelligence and Translation Laboratory, Harbin Institute of Technology, Harbin, China 21s103280@stu.hit.edu.cn, 21s103127@stu.hit.edu.cn, 21s003015@stu.hit.edu.cn, hitxb@hit.edu.cn Abstract This paper describes the system for the Semeval-2022 Task4 ”Patronizing and Conde- scending Language Detection”.An entity en- gages in Patronizing and Condescending Lan- guage(PCL) when its language use shows a superior attitude towards others or depicts them in a compassionate way. The task contains two parts. The first one is to identify whether the sentence is PCL, and the second one is to cate- gorize PCL. Through experimental verification, the RoBERTa-based model will be used in our system. Respectively, for subtask 1, that is, to judge whether a sentence is PCL, the method of retraining the model with specific task data is adopted, and the method of splicing [CLS] and the keyword representation of the last three layers as the representation of the sentence; for subtask 2, that is, to judge the PCL type of the sentence, in addition to using the same method as task1, the method of selecting a special loss for Multi-label text classification is applied. We give a clear ablation experiment and give the effect of each method on the final result. Our project ranked 11th out of 79 teams partici- pating in subtask 1 and 6th out of 49 teams participating in subtask 2. 1 Introduction The effect of Patronizing and Condescending Lan- guage (PCL) towards vulnerable communities in the media is not always conscious and the intention of the author is often to help the person or group they refer to (e.g. by raising awareness or funds or moving the audience to action). However, these su- perior attitudes and discourse of pity can routinize discrimination and make it less visible. While there has been substantial work on modeling language that purposefully undermines others, the modeling of PCL is still an emergent area of study in NLP since PCL is the speaker’s unconscious superior speaking attitude, the special word that causes PCL is subtle compared to the keywords in other natural language processing problems. The authors decided to evaluate the questions separately. In Semeval-2022 task 4: Patroniz- ing and Condescending Language Detection(P´erez- Almendros et al., 2022), the purpose of subtask 1 is to identify whether a sentence is PCL. In contrast, the goal of subtask 2 is to indicate the presence of PCL at the text span level, which detects the exact categories in the seven categories of PCL.In sub- task 1, the method of using data set retraining to make the pre-trained language model learn the spe- cific distribution of the data set, adding keywords to the input, and integrating five RoBERTa-based models, subtask 2 is to select k from 7 For classifi- cation tasks, task-specific loss calculation methods are designed. These methods will be explained in …

Content truncated to 500 words