Amherst685 at SemEval-2021 Task 7: Joint Modeling of Classification and Regression for Humor and Offense

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 1190–1195 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 1190 Amherst685 at SemEval-2021 Task 7: Joint Modeling of Classification and Regression for Humor and Offense Brian Zylich Akshay Gugnani Gabriel Brookman Nicholas Samoray University of Massachusetts Amherst {bzylich,agugnani,gbrookman,nsamoray}@umass.edu Abstract This paper describes our submission to the SemEval’21: Task 7- HaHackathon: Detect- ing and Rating Humor and Offense. In this challenge, we explore intermediate finetun- ing, backtranslation augmentation, multitask learning, and ensembling of different language models. Curiously, intermediate finetuning and backtranslation do not improve perfor- mance, while multitask learning and ensem- bling do improve performance. We explore why intermediate finetuning and backtransla- tion do not provide the same benefit as on other natural language processing tasks and offer in- sight into the errors that our model makes. Our best performing system ranks 7th on Task 1b with an RMSE of 0.5339. 1 Introduction With the advancement in deep learning methods, NLP tasks like sentiment analysis and opinion min- ing have achieved high accuracy, however detection of some salient forms of figurative language such as humor remain difficult tasks. Being able to infer humor and offense with a high accuracy can help improve and lead to better performance on downstream applications, such as content moderation, sentiment analysis, etc. This would be useful for various downstream applica- tions, such as understanding tweets, reviews and feedback. However, humor detection is not trivial. What makes identifying humor hard? Humor can consist of styles ranging from sarcastic to slap- stick comedy, and it factors in both individual pref- erences and underlying cultures. Context, sounds, and vision or any combination of these can be key in building to a punchline (Cai, 2019). Humor ap- preciation is also highly subjective, as age, gender, and socio-economic status often impact the percep- tion of a joke. Meaney (2020) identify 3 ways that classification of humor is difficult: (1) Humor can differ between cultures, (2) Humor can also differ within cultures, and (3) Humor differs within the same person. Our contributions are as follows: (1) we explore whether intermediate finetuning on other humor and offense datasets is helpful for this task, (2) we seek to identify if backtranslation augmentation is useful for humor detection, (3) we show that multitask learning is helpful when classifying and scoring both humor and offense, and (4) we find that ensembling different language models leads to improved results on some tasks. The code for our experiments is available at our Github repository1. 2 Related Work The challenge of humor detection has gained trac- tion since 2017. Meaney (2020), explains in their proposal that prior work has explored humor detec- tion as an objective task, averaging all annotations for a joke, to produce a single classification or rat- ing. This treats humor as an objective concept, which is not the case. This motivated their chal- lenge for SemEval’21 (Meaney et al., 2021) to ex- plore these dimensions of humor and offense. We discuss …

Content truncated to 500 words