UoR at SemEval-2021 Task 7: Utilizing Pre-trained DistilBERT Model and Multi-scale CNN for Humor Detection

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 1179–1184 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 1179 UoR at SemEval-2021 Task 7: Utilizing Pre-trained DistilBERT Model and Multi-scale CNN for Humor Detection Zehao Liu, Carl Haines, Huizhi Liang University of Reading White Knights, Berkshire, RG6 6AH United Kingdom z.liu3@pgr.reading.ac.uk, carl.haines@pgr.reading.ac.uk, huizhi.liang@reading.ac.uk Abstract Humor detection is an interesting but difficult task in NLP. Humor might not be obvious in text because it may be embedded into con- text, hide behind the literal meaning of the phrase and require prior knowledge to under- stand. We explored different shallow and deep methods to create a humour detection classi- fier for task 7-1a. Models like Logistic Re- gression, LSTM, MLP, CNN were used, and pre-trained models like DistilBERT were in- troduced to generate accurate vector represen- tation for textual data. We focused on apply- ing a multi-scale strategy on modelling, and compared different models. Our best model is the DistilBERT+MultiScale CNN which used different sizes of CNN kernel to get multi- ple scales of features. This method achieved 93.7% F1-score and 92.1% accuracy on the test set. 1 Introduction Humor detection is an interesting but difficult task in Natural language processing (NLP) and requires various techniques to understand the meaning of a sentence and identify humor. For example, humour by sarcasm can mean that a piece of text can have two very different meanings and the NLP algorithm needs to be able to understand which meaning is intended. The aim of task 7-1a of SemEval 2021 (Meaney et al., 2021) was to address the challenge of clas- sifying humour in text. Provided for this task was a dataset constructed of short phrases in English along with a label classifying whether or not each phrase is intended to be humorous. The labels have been obtained by surveying a group of people that represent a variety of genders, political stances and income levels. The given dataset includes training set with 8000 labeled sentences, a development set of 1000 sentences and a test set of 1000 sentences. The dataset was made up of English phrases that were labeled by their intent to be humorous. This means the label annotators were not saying whether or not they found the text funny but whether the writer of the text intended it to be funny. The texts were predominantly one sentence long with a small proportion being two or three sentences. Each phrase was labeled for humour intent and, if humour was intended, then a rating was given for how funny it is. Also of the texts intended to be humorous, a label was given for whether it is offensive, and if so, how offensive it is. The texts covered a range of types of jokes such as puns, sarcasm, dark jokes and “Dad” jokes. One example is “I never finish anything. I have a black belt in partial arts.” which contains a pun and is labeled as humorous. Our best approach is …

Content truncated to 500 words