LMML at SemEval-2020 Task 7: Siamese Transformers for Rating Humor in Edited News Headlines

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1026–1032 Barcelona, Spain (Online), December 12, 2020. 1026 LMML at SemEval-2020 Task 7: Siamese Transformers for Rating Humor in Edited News Headlines Pramodith Ballapuram Independent Researcher, Atlanta GA, USA pramodith1@gmail.com Abstract This paper contains a description of my solution to the problem statement of SemEval 2020: Assessing the Funniness of Edited News Headlines. I propose a Siamese Transformer based approach, coupled with a Global Attention mechanism that makes use of contextual embeddings and focus words, to generate important features that are fed to a 2 layer perceptron to rate the funniness of the edited headline. I detail various experiments to show the performance of the system. The proposed approach outperforms a baseline Bi-LSTM architecture and finished 5th (out of 49 teams) in sub-task 1 and 4th (out of 32 teams) in sub-task 2 of the competition and was the best non-ensemble model in both tasks. 1 Introduction Machines that can recognize and understand humor can prove to be invaluable in applications like chat bots, personal digital assistants in order to make communication more fun and humane, story and script generation to provide comical relief or even in recommendation engines that can provide better recommendations to people on what Netflix stand-up show they can watch next. At the end of the day we all live to laugh don’t we? Surprisingly enough, there hasn’t been much work in the field of AI along these lines. The organizers of SemEval-2020 Task 7 (Hossain et al., 2020a) released a new dataset in the English language and created a couple of sub tasks that can hopefully take us a step forward in creating machines that better understand humor. HAHA - Humor Analysis based on Human Annotation (Castro et al., ), (Chiruzzo et al., 2019) started in 2018 was a similar task, where the dataset comprised of Spanish Tweets and participants were asked to classify the tweets as either a joke or not and also rate the jokes on a scale of 0-5. In their overview paper, the authors describe that teams that used Transformer Based Models such as BERT (Devlin et al., 2019) and ULMFit (Howard and Ruder, 2018), along with techniques like slanted learning rates, domain specific language modeling etc. proved valuable. My approach takes inspiration from this and focuses on Transformer Models. SemEval 2017 Task-6 (Potash et al., 2017) consisted of sub-tasks asking participants to rank the funniness of tweets that had a specific HashTag. They mention that some of the top teams used Siamese Networks (Bromley et al., 1994), (Koch, 2015) based approaches. He et. al., (2019) introduce the concepts of local surprisal and global surprisal their work suggests that for a pun to be considered good, the pun-word must have high agreeableness in the local context of where it occurs in the sentence but a lower level of agreeableness in the global context of the entire sentence. I take inspiration from this idea as well. My hypothesis is that an edit …

Content truncated to 500 words