PINGAN Omini-Sinitic at SemEval-2022 Task 4: Multi-prompt Training for Patronizing and Condescending Language Detection

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 313 - 318 July 14-15, 2022 ©2022 Association for Computational Linguistics PINGAN Omini-Sinitic at SemEval-2022 Task 4: Multi-prompt Training for Patronizing and Condescending Language Detection Ye Wang Yanmeng Wang Baishun Ling Zexiang Liao Shaojun Wang Jing Xiao Ping An Technology, Beijing 100191, China {wangye430,wangyanmeng219,lingbaishun271,liaozexiang901, wangshaojun851,xiaojing661}@pingan.com.cn Abstract This paper describes the second-placed system for subtask 2 and the ninth-placed system for subtask 1 in SemEval 2022 Task 4: Patroniz- ing and Condescending Language Detection. We propose an ensemble of prompt training and label attention mechanism for multi-label classification tasks. Transfer learning is intro- duced to transfer the knowledge from binary classification to multi-label classification. The experimental results proved the effectiveness of our proposed method. The ablation study is also conducted to show the validity of each technique. 1 Introduction Patronizing and Condescending Language (PCL) is proposed by Pérez-Almendros et al. (2020), which builds a dataset for PCL detection. The Patronizing and Condescending Language Detection (Pérez- Almendros et al., 2022) contains two text classifi- cation tasks. Subtask 1 is a binary classification task, which requires a system to predict whether the paragraph contains any form of PCL. Subtask 2 is a multi-label classification task, which must identify which PCL categories express the condescension. PCL Detection is a sentiment analysis task and we treated subtask 2 as an Aspect-Based Sentiment Analysis (ABSA) (Jo and Oh, 2011) task, which is also a multi-label classification to classify the sentence sentiment on different aspects. Early works on ABSA focus on feature engineer- ing (Wagner et al., 2014) and subsequent neural network-based methods (Wang et al., 2017). Re- cently, Pre-trained Language Models (LMs) such as BERT (Devlin et al., 2019), ROBERTA (Liu et al., 2019), ALBERT (Lan et al., 2020) have been proposed and brought significant improvement in various NLP applications. As there is not much improvement in ABSA task through direct imple- mentation of those pre-trained LMs, Bert-pair (Sun et al., 2019) has been proposed to help them adapt to ABSA effectively. Furthermore, Bu et al. (2021) proposed an attention between sentence embed- dings and label embeddings to focus on the crucial tokens which are related to the label. Recently, CapsNet-Bert (Jiang et al., 2019) used the atten- tion between the label and the input with capsule network to improve the performance. Nowadays, the Pattern-Exploiting Training (PET) (Schick and Schütze, 2021) and P-tuning (Liu et al., 2021) has been proposed to utilize the pre-trained LMs more effectively, which trains fine- tune model by the pre-training tasks such as Mask Language Modeling (MLM) by adding a prompt sequence to the input. Considering the similarity between PCL and ABSA, we applied PET and Bert-pair in PCL De- tection through the following steps: Firstly, we treat the subtask 1 as a prompt training task (as described in PET) by using a PCL description as the prompt. Secondly, we transform the subtask 2 from a multi-label classification task to multiple binary classification tasks by using the label names as …

Content truncated to 500 words