“So You Think You’re Funny?”: Rating the Humour Quotient in Standup Comedy

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10073–10079 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 10073 “So You Think You’re Funny?”: Rating the Humour Quotient in Standup Comedy Anirudh Mittal†, Pranav Jeevan⋄, Prerak Gandhi♣, Diptesh Kanojia‡, Pushpak Bhattacharyya⋆ †,⋄,♣,⋆Indian Institute of Technology Bombay, Mumbai ‡Centre for Translation Studies, University of Surrey, United Kingdom †,♣,⋆{anirudhmittal,prerakgandhi,pb}@cse.iitb.ac.in ⋄pranavjp@ee.iitb.ac.in ‡d.kanojia@surrey.ac.uk Abstract Computational Humour (CH) has attracted the interest of Natural Language Processing and Computational Linguistics communities. Cre- ating datasets for automatic measurement of humour quotient is difficult due to multiple possible interpretations of the content. In this work, we create a multi-modal humour- annotated dataset (∼40 hours) using stand-up comedy clips. We devise a novel scoring mechanism to annotate the training data with a humour quotient score using the audience’s laughter. The normalized duration (laughter duration divided by the clip duration) of laugh- ter in each clip is used to compute this humour coefficient score on a five-point scale (0-4). This method of scoring is validated by compar- ing with manually annotated scores, wherein a quadratic weighted kappa of 0.6 is obtained. We use this dataset to train a model that pro- vides a “funniness” score, on a five-point scale, given the audio and its corresponding text. We compare various neural language models for the task of humour-rating and achieve an accu- racy of 0.813 in terms of Quadratic Weighted Kappa (QWK). Our “Open Mic” dataset is re- leased for further research along with the code. 1 Introduction Humour is one of the most important lubricants of communication between people. Humour is subjec- tive and, at times, also requires cultural knowledge as humour is often dependent on stereotypes in a culture or a country. At times, even cultural ap- propriation is used to convey humour, which can be offensive to minority cultures1 (Rosenthal et al., 2015; Kuipers, 2017). The factors listed above, along with the underlying subjectivity in humour render the task of rating humour, difficult for ma- chines (Meaney, 2020). The task of humour classi- fication suffers due to this subjectivity and the lack of datasets that rate the “funniness” of content. †Corresponding Author 1Racism in Comedy: An opinion piece. In this paper, we propose rating humour on a scale of zero to four. We create the first multi- modal dataset2 using standup comedy clips and compute the humour quotient of each clip using the audience laughter. The validity of our scoring criteria is verified by finding the overall agreement between human annotation and automated scores. We use the audio and text-based signals to process this multi-modal data to generate ‘humour ratings’. Since humour annotation is subjective, even the data annotated by humans might not provide an objective measure. We reduce this subjectivity by taking laughter feedback from a larger audience. To the best of our knowledge, no previous literature has proposed an automatically humour-rated multi- modal dataset and used it in ML model-building to automatically obtain the humor score. Standup comedy is an art form …

Content truncated to 500 words