abcbpc at SemEval-2021 Task 7: ERNIE-based Multi-task Model for Detecting and Rating Humor and Offense

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 286–289 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 286 abcbpc at SemEval-2021 Task 7: ERNIE-based Multi-task Model for Detecting and Rating Humor and Offense Chao Pang, Xiaoran Fan, Weiyue Su, Xuyi Chen Shuohuan Wang, Jiaxiang Liu, Xuan Ouyang Shikun Feng, Yu Sun Baidu Inc., China {pangchao04,fanxiaoran,suweiyue,chenxuyi}@baidu.com {wangshuohuan,liujiaxiang,ouyangxuan}@baidu.com {fengshikun01,sunyu02}@baidu.com Abstract This paper describes our system participated in Task 7 of SemEval-2021: Detecting and Rating Humor and Offense. The task is de- signed to detect and score humor and of- fense which are influenced by subjective fac- tors. In order to obtain semantic information from a large amount of unlabeled data, we ap- plied unsupervised pre-trained language mod- els. By conducting research and experiments, we found that the ERNIE 2.0 and DeBERTa pre-trained models achieved impressive perfor- mance in various subtasks. Therefore, we ap- plied the above pre-trained models to fine-tune the downstream neural network. In the process of fine-tuning the model, we adopted multi- task training strategy and ensemble learning method. Based on the above strategy and method, we achieved RMSE of 0.4959 for sub- task 1b, and finally won the first place. 1 Introduction Humor, as a highly subjective phenomenon, can be affected by various factors. Automatic humor recognition relies on annotated data to determine whether the text is humorous or not (Mihalcea and Strapparava, 2005). However, such a binary clas- sification does not capture the level of humor, so assessing the level of humor is of great significance (Garimella et al., 2020). Since humor can be influ- enced by many factors, such as age, and may offend others. Based on such a situation, SemEval-2021 Task 7 focuses on linking humor and offense across different age groups (Meaney et al., 2021). But there are still many challenges to this task. For ex- ample, the dataset for the task is small and the texts are short, which does not allow for adequate train- ing. To address these issues, we utilized unsuper- vised pre-trained language models and fine-tuned these models for specific downstream subtasks. Af- ter conducting research and extensive comparative experiments, the results show that ERNIE 2.0 (Sun et al., 2019b) and DeBERTa pre-trained models performed best on the subtasks. These large un- supervised language models were pre-trained on a large amount of unlabeled data to extract valuable lexical, syntactic, and semantic information from the corpus. Vector representations of text com- puted by these models are applied to fine-tune the downstream neural networks for the subtasks. The multi-task training and ensemble learning method significantly improve our model’s performance. The rest of the paper is organized as follows: Section 2 provides a brief description of the re- lated work, and Section 3 describes our proposed approach in detail. In Section 4, the experiments are described in detail and the results are presented. Finally, we summarize the whole paper and discuss future research directions in Section 5. 2 Related Work In the early research of humor and offense …

Content truncated to 500 words