QUB at SemEval-2017 Task 6: Cascaded Imbalanced Classification for Humor Analysis in Twitter

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 380–384, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics QUB at SemEval-2017 Task 6: Cascaded Imbalanced Classification for Humor Analysis in Twitter Xiwu Han and Gregory Toner Queen’s University Belfast Belfast, BT7 1NN, UK x.han@qub.ac.uk; g.toner@qub.ac.uk Abstract This paper presents our submission to SemEval-2017 Task 6: #HashtagWars: Learning a Sense of Humor. There are two subtasks: A. Pairwise Comparison, and B. Semi-Ranking. Our assumption is that the distribution of humorous and non-humor- ous texts in real life language is naturally imbalanced. Using Naïve Bayes Multino- mial with standard text-representation fea- tures, we approached Subtask B as a se- quence of imbalanced classification prob- lems, and optimized our system per the macro-average recall. Subtask A was then solved via the Semi-Ranking results. On the final test, our system was ranked 10th for Subtask A, and 3rd for Subtask B. 1 Introduction Humor is an essential trait of human intelligence that has not yet been addressed extensively in cur- rent AI research1. It’s certainly one of the most in- teresting and puzzling research areas in the field of natural language understanding, and developing techniques that enable computers to understand humor in human languages deserves research at- tention (Yang et al., 2015). Humor recognition or analysis by computers aims to determine whether a sentence in context expresses a certain degree of humor. This can be extremely challenging (Attardo, 1994) because no universal definition of humor has been achieved, humor is highly contextual, and there are many different types of humor with different character- istics (Raz, 2012). Previous studies (Mihalcea and 1 http://alt.qcri.org/semeval2017/task6/ Strapparava, 2005; Yang et al., 2015; Zhang and Liu, 2014; Purandare and Litman, 2006; Bertero and Fung, 2016) dealt with the humor recognition task as a binary classification task, which was to categorize a given text as humorous or non-humor- ous (Li et al., 2016). Textual data consisting of comparable amounts of humorous texts and non- humorous texts were collected, and a classification model was then built using textual features. Barb- ieri and Saggion (2014) examined cross-domain application of humor detection using Twitter data. Purandare and Litman (2006) used data from a fa- mous TV series, Friends. Speakers’ turns which occurred right before simulated laughter were de- fined as humorous ones and the other turns as non- humorous ones. They also used speakers’ acoustic characteristics as features. Bertero and Fung (2016) pursued a similar hypothesis. Their target was to categorize an utterance in a sitcom, The Big Bang Theory, into those followed by laughter or not. They were the first to use a deep learning al- gorithm for humor classification. Besides, because genre bias can be problematic, Yang et al. (2015) tried to minimize genre differences between hu- morous and non-humorous texts. SemEval-2017 Task 6 aims to encourage the de- velopment of methods that should take into account the continuous nature of humor, on the one hand, and to characterize the sense of humor of a …

Content truncated to 500 words