Humor Detection: A Transformer Gets the Last Laugh

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 3621–3625, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 3621 Humor Detection: A Transformer Gets the Last Laugh Orion Weller Computer Science Department Brigham Young University orionw@byu.edu Kevin Seppi Computer Science Department Brigham Young University kseppi@byu.edu Abstract Much previous work has been done in attempt- ing to identify humor in text. In this pa- per we extend that capability by proposing a new task: assessing whether or not a joke is humorous. We present a novel way of ap- proaching this problem by building a model that learns to identify humorous jokes based on ratings gleaned from Reddit pages, consist- ing of almost 16,000 labeled instances. Using these ratings to determine the level of humor, we then employ a Transformer architecture for its advantages in learning from sentence context. We demonstrate the effectiveness of this approach and show results that are com- parable to human performance. We further demonstrate our model’s increased capabili- ties on humor identification problems, such as the previously created datasets for short jokes and puns. These experiments show that this method outperforms all previous work done on these tasks, with an F-measure of 93.1% for the Puns dataset and 98.6% on the Short Jokes dataset. 1 Introduction Recent advances in natural language processing and neural network architecture have allowed for widespread application of these methods in Text Summarization (Liu et al., 2018), Natural Lan- guage Generation (Bahuleyan, 2018), and Text Classification (Yang et al., 2016). Such advances have enabled scientists to study common language practices. One such area, humor, has garnered fo- cus in classification (Zhang and Liu, 2014; Chen and Soo, 2018), generation (He et al., 2019; Vali- tutti et al., 2013), and in social media (Raz, 2012). The next question then is, what makes a joke humorous? Although humor is a universal con- struct, there is a wide variety between what each individual may find humorous. We attempt to fo- cus on a subset of the population where we can quantitatively measure reactions: the popular Red- dit r/Jokes thread. This forum is highly popu- lar - with tens of thousands of jokes being posted monthly and over 16 million members. Although larger joke datasets exist, the r/Jokes thread is un- paralleled in the amount of rated jokes it contains. To the best of our knowledge there is no compa- rable source of rated jokes in any other language. These Reddit posts consist of the body of the joke, the punchline, and the number of reactions or up- votes. Although this type of humor may only be most enjoyable to a subset of the population, it is an effective way to measure responses to jokes in a large group setting.1 What enables us to perform such an analysis are the recent improvements in neural network ar- chitecture for natural language processing. These breakthroughs started with the Convolutional Neu- ral …

Content truncated to 500 words