McRock at SemEval-2022 Task 4: Patronizing and Condescending Language Detection using Multi-Channel CNN, Hybrid LSTM, DistilBERT and XLNet

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 409 - 417 July 14-15, 2022 ©2022 Association for Computational Linguistics McRock at SemEval-2022 Task 4: Patronizing and Condescending Language Detection using Multi-Channel CNN, Hybrid LSTM, DistilBERT and XLNet Marco Siino, Marco La Cascia, and Ilenia Tinnirello Università degli Studi di Palermo, Palermo, Italy {marco.siino, marco.lacascia, ilenia.tinnirello}@unipa.it Abstract In this paper we propose four deep learning models for the task of detecting and classify- ing Patronizing and Condescending Language (PCL) using a corpus of over 13,000 annotated paragraphs in English. The task, hosted at SemEval-2022, consists of two different sub- tasks. The Subtask 1 is a binary classification problem. Namely, given a paragraph, a sys- tem must predict whether or not it contains any form of PCL. The Subtask 2 is a multi-label classification task. Given a paragraph, a system must identify which PCL categories express the condescension. A paragraph might contain one or more categories of PCL. To face with the first subtask we propose a multi-channel Convolutional Neural Network (CNN) and an Hybrid LSTM. Using the multi-channel CNN we explore the impact of parallel word emebed- dings and convolutional layers involving differ- ent kernel sizes. With Hybrid LSTM we focus on extracting features in advance, thanks to a convolutional layer followed by two bidirec- tional LSTM layers. For the second subtask a Transformer BERT-based model (i.e. Dis- tilBERT) and an XLNet-based model are pro- posed. The multi-channel CNN model is able to reach an F1 score of 0.2928, the Hybrid LSTM model is able to reach an F1 score of 0.2815, the DistilBERT-based one an average F1 of 0.2165 and the XLNet an average F1 of 0.2296. In this paper, in addition to system descriptions, we also provide further analysis of the results, highlighting strengths and limita- tions. We make all the code publicly available and reusable on GitHub1. 1 Introduction With the exponential growth of contents shared on social networks, a lot of new challenging tasks have emerged. Many are currently studied and addressed by scholars, and a pletora of novel machine learn- ing approaches have been proposed (Arpaci et al., 1https://github.com/marco-siino/ McRock-SemEval-2022-Task4 2021), (Hosseinalipour and Ghanbarzadeh, 2022), (Siino et al., 2020). Some of the most common tasks, often co-located with international confer- ences, are those about fake news (Rangel et al., 2020), hate speech (Bosco et al., 2018), misogyny (Fersini et al., 2018) and cyberbulling (Kumar et al., 2018) detection. For these purposes there is a constantly growing need for tools that can automatically extract and classify information from online feeds, to face with consolidated as well as with emerging social issues. Interest in Natural Language Processing (NLP) has increased in recent years with advances in machine and deep learning architectures. There have been significant efforts in developing methods to auto- matically detect and classify text content available online nowadays. Together with the already mentioned tasks, an emerging one is about detecting Patroniz- ing and Condescending Language (PCL) (Pérez- Almendros et al., 2020). The PCL …

Content truncated to 500 words