JokeMeter at SemEval-2020 Task 7: Convolutional Humor

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 843–851 Barcelona, Spain (Online), December 12, 2020. 843 JokeMeter at SemEval-2020 Task 7: Convolutional humor Martin Docekal, Martin Fajcik, Josef Jon, Pavel Smrz Brno University of Technology, Faculty of Information Technology 612 66 Brno, Czech Republic {idocekal,ifajcik,ijon,smrz}@fit.vutbr.cz Abstract This paper describes our system that was designed for Humor evaluation within the SemEval-2020 Task 7. The system is based on convolutional neural network architecture. We investigate the system on the official dataset, and we provide more insight to model itself to see how the learned inner features look. 1 Introduction This paper deals with estimating the humor of edited English news headlines (Hossain et al., 2020b; Hossain et al., 2020a). The illustration of tasks is in Figure 1. The original text sequence is given, which represents a title, with the annotated part that is edited along with the edit itself. Our responsibility is to determine how funny this change is in the range from 0 to 3 (inclusive). This is called Sub-Task 1. We also participate in the Sub-Task 2, in that we should decide which from both given edits is the funnier one. For the second task, we used the approach of reusing the model from the first task as it is described in section 4. So we are focusing the description on Sub-Task 1. Kushner to visit Mexico following latest Trump tirades therapist 2.8 Sub-Task 1 Sub-Task 2 Kushner to visit Mexico following latest Trump tirades therapist Oklahoma isn't working. Can anyone �x this failing American state? ok Which is funnier? mean grade edit Figure 1: Examples for both tasks. The red color in the original title marks part that is substituted with a green word. Official results were achieved with a Convolutional Neural Networks (CNNs) (LeCun et al., 1999; Fukushima and Miyake, 1982), but we also tested numerous other approaches such as SVM (Cortes and Vapnik, 1995) and pre-trained transformer model (Vaswani et al., 2017). The humor is a very subjective phenomenon, as can be seen from the inter-agreement on label annotation in Sub-Task 1 dataset1. The given data labels do not allow us to learn a sense of humor of a human annotator because the dataset does not specify from whom the grade comes. So, for example, if we have one annotator that likes dark humor and all the others not, we will be considering such a replacement as not humorous no meter if it is excellent dark humor or not. In other words, we may say that we are searching for some most common kind of humor. The dominant theory of humor is the Incongruity Theory (Morreall, 2016). It says that we are finding humor in perceiving something unexpected (incongruous) that violates expectations that were set up by the joke. There are samples, in the provided dataset, that uses the incongruity to create humor. Moreover, according to Hossain et al. (2020a), we can see a positive influence of incongruity on systems results for the dataset. 1The …

Content truncated to 500 words