CS-UM6P at SemEval-2021 Task 7: Deep Multi-Task Learning Model for Detecting and Rating Humor and Offense

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 1135–1140 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 1135 CS-UM6P at SemEval-2021 Task 7: Deep Multi-Task Learning Model for Detecting and Rating Humor and Offense Kabil Essefar1 Abdellah El Mekki1 Abdelkader El Mahdaouy1 Nabil El Mamoun2 Ismail Berrada1 1School of Computer Sciences, Mohammed VI Polytechnic University, Morocco 2Faculty of Sciences Dhar EL Mahraz, Sidi Mohamed Ben Abdellah University, Morocco {firstname.lastname}@um6p.ma Abstract Humor detection has become a topic of inter- est for several research teams, especially those involved in socio-psychological studies, with the aim to detect the humor and the temper of a targeted population (e.g. a community, a city, a country, the employees of a given company). Most of the existing studies have formulated the humor detection problem as a binary classi- fication task, whereas it revolves around learn- ing the sense of humor by evaluating its dif- ferent degrees. In this paper, we propose an end-to-end deep Multi-Task Learning (MTL) model to detect and rate humor and offense. It consists of a pre-trained transformer encoder and task-specific attention layers. The model is trained using MTL uncertainty loss weight- ing to adaptively combine all sub-tasks objec- tive functions. Our MTL model tackles all sub- tasks of the SemEval-2021 Task-7 in one end- to-end deep learning system and shows very promising results. 1 Introduction Humor is a human trait that defines the emotional and behavioral characteristics of an individual. It refers to the quality of being amusing, comic, sar- castic, etc. Most dictionaries define humor also as a message, whose ingenuity or verbal skill, or incongruity, that has the power to make individual laughing. Humor and offensive language detection tasks are increasingly becoming hot research topics in Natural Language Processing (NLP) (Zampieri et al., 2019; Reyes et al., 2012; Gleason et al., 2019; van den Beukel and Aroyo, 2018; Cattle and Ma, 2018; Singh et al., 2020). Existing research works have tackled humor detection as either a binary classification problem (Weller and Seppi, 2019; Annamoradnejad, 2020) or a ranking task (Potash et al., 2017; Hossain et al., 2020; Zhang et al., 2019). Similarly, most research works on offensive language detection have proposed meth- ods and approaches to discriminate between offen- sive and not-offensive texts (Zampieri et al., 2019, 2020), whereas, other research works have classi- fied offensive content into more fine-grained levels (Wiegand et al., 2018; Kumar et al., 2018; Risch et al., 2020). Fine-tuning pre-trained transformer-based lan- guage models on the target task data has shown state-of-the-art (SOTA) results in many NLP ap- plications (Devlin et al., 2019; Liu et al., 2019). For instance, several research works on humor and offensive language detection have achieved SOTA performances using pre-trained transformer-based language models (Zampieri et al., 2019; Weller and Seppi, 2019; Zampieri et al., 2020). In this paper, we describe our system submit- ted to the SemEval-2021 Task-7 (Sub-Tasks 1 and 2) (Meaney et al., 2021). We propose an end-to- end deep Multi-Task Learning (MTL) model …

Content truncated to 500 words