CHoRaL: Collecting Humor Reaction Labels from Millions of Social Media Users

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4429–4435 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 4429 CHoRaL: Collecting Humor Reaction Labels from Millions of Social Media Users Zixiaofan Yang and Shayan Hooshmand and Julia Hirschberg Department of Computer Science Columbia University New York, NY, USA {zy2231,shayan.hooshmand}@columbia.edu,julia@cs.columbia.edu Abstract Humor detection has gained attention in re- cent years due to the desire to understand user- generated content with figurative language. However, substantial individual and cultural differences in humor perception make it very difficult to collect a large-scale humor dataset with reliable humor labels. We propose CHoRaL, a framework to generate perceived humor labels on Facebook posts, using the nat- urally available user reactions to these posts with no manual annotation needed. CHoRaL provides both binary labels and continuous scores of humor and non-humor. We present the largest dataset to date with labeled humor on 785K posts related to COVID-19. Addi- tionally, we analyze the expression of COVID- related humor in social media by extracting lexico-semantic and affective features from the posts, and build humor detection models with performance similar to humans. CHoRaL en- ables the development of large-scale humor de- tection models on any topic and opens a new path to the study of humor on social media. 1 Introduction Humor is ubiquitous –– it forms a crucial part of people’s lives both online and off. Automatically detecting humor, then, has become an important task, with applications from misinformation to ad- vertising to philosophy. From a psychological per- spective, humor represents anything people say or do that others perceive as funny and tends to make them laugh (Martin, 2010). Humor percep- tion, though, is highly individualistic (Ruch, 2001), making it hard to reliably annotate humor. Researchers have proposed various methods to collect humorous and non-humorous data with min- imal annotation needed. Most attempts have fo- cused on distinguishing between jokes and news, which both have natural labels on humor and can be scraped automatically. This large stylistic dif- ference makes detecting humor easier –– but it is Figure 1: User reactions to a humorous Facebook post (top) and a non-humorous post (bottom). far from most real-world scenarios where humor- ous and non-humorous texts come from the same domain. Another technique collects social media posts by humor- and non-humor-related hashtags, but this method suffers from large data noise and low labeling accuracy (Zhang and Liu, 2014). Fi- nally, there have been studies to use the number of Reddit upvotes as humor labels (Weller and Seppi, 2019, 2020). Though this technique sources data from the same domain, that domain is too limited in scope: all the data comes from one single subred- dit. This specificity means that the data represents only the humor perception of a particular group of Reddit users, dedicated to producing witty jokes. To address these problems of specificity and domain discrepancy in humorous data collection, we propose CHoRaL, a framework for Collecting Humor Reaction Labels. CHoRaL generates per- ceived humor scores using the …

Content truncated to 500 words