YNU-HPCC at SemEval-2022 Task 4: Finetuning Pretrained Language Models for Patronizing and Condescending Language Detection

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 454 - 458 July 14-15, 2022 ©2022 Association for Computational Linguistics YNU-HPCC at SemEval-2022 Task 4: Finetuning Pretrained Language Models for Patronizing and Condescending Language Detection Wenqiang Bai, Jin Wang and Xuejie Zhang School of Information Science and Engineering Yunnan University Kunming, China Contact: ynubwq@mail.ynu.edu.cn, {wangjin, xjzhang}@ynu.edu.cn Abstract This paper describes a system built for the SemEval-2022 competition. As participants in Task 4: Patronizing and Condescending Lan- guage Detection, we implemented the text sen- timent classification system for two subtasks in English. Both subtasks involve determin- ing emotions; subtask 1 requires us to deter- mine whether the text belongs to the PCL cate- gory (single-label classification), and subtask 2 requires us to determine to which PCL cate- gory the text belongs (multi-label classifica- tion). Our system is based on the bidirec- tional encoder representations from transform- ers (BERT) model. For the single-label classifi- cation, our system applies a BertForSequence- Classification model to classify the input text. For the multi-label classification, we use the fine-tuned BERT model to extract the sentiment score of the text and a fully connected layer to classify the text into the PCL categories. Our system achieved relatively good results on the competition’s official leaderboard. 1 Introduction Text classification is an area of natural language processing (NLP) that aims to classify text using certain features. Previous studies on text classifica- tion tasks used traditional machine learning meth- ods, which require researchers to manually design features. Feature extraction methods such as term frequency–inverse document frequency (TF-IDF) (Hakim et al., 2014) and N-Gram (Cavnar et al., 1994) are used to extract features from original documents, and then the features are input into classifiers such as naive Bayes(Berrar, 2019), sup- port vector machines (SVMs) (Hearst et al., 1998), and decision trees (Vens et al., 2008). Since the advent of deep learning, text classification tasks are achievable without manual extraction of text features. Researchers must simply pretreat the text and incorporate it into a deep learning model for training. For text classification using deep learn- ing methods, the classification accuracy is often higher than that of traditional machine learning methods. With their continuous improvement, deep learning models, such as recurrent neural networks (RNNs)(Zaremba et al., 2014), multi-channel CNN- LSTM (Zhang et al., 2017),gate recurrent units (GRUs) (Rana, 2016), long short-term memory (LSTM) (Shi et al., 2015), bidirectional long short- term memory (Bi-LSTM) (Zhang et al., 2015), and attention-based Bi-LSTM (Zhang et al., 2018) net- works, can be used to solve text classification prob- lems. In recent years, bidirectional encoder repre- sentations from transformers (BERT) (Devlin et al., 2018), a new deep learning model, has achieved, or even surpassed, human performance in multi- ple tasks within the NLP domain, including text classification. Task 4 of the SemEval-2022 consists of the fol- lowing two subtasks. • Subtask 1: identifying whether the sentence contains any kind of PCL. • Subtask 2: identifying which types of PCL the sentence contains. In this paper, we …

Content truncated to 500 words