Unified Humor Detection Based on Sentence-pair Augmentation and Transfer Learning

Unified Humor Detection Based on Sentence-pair Augmentation and Transfer Learning Minghan Wang1, Hao Yang1, Ying Qin1, Shiliang Sun2, Yao Deng1 Huawei Translation Service Center, Beijing, China East China Normal University, Shanghai, China {wangminghan, yanghao30, qinying, dengyao3}@huawei.com slsun@cs.ecnu.edu.cn Abstract We propose a unified multilingual model for humor detection which can be trained under a transfer learning framework. 1) The model is built based on pre-trained multilingual BERT, thereby is able to make predictions on Chinese, Russian and Span- ish corpora. 2) We step out from sin- gle sentence classification and propose sequence-pair prediction which considers the inter-sentence relationship. 3) We pro- pose the Sentence Discrepancy Prediction (SDP) loss, aiming to measure the seman- tic discrepancy of the sequence-pair, which often appears in the setup and punchline of a joke. Our method achieves two SoTA and a second-place on three humor detec- tion corpora in three languages (Russian, Spanish and Chinese), and also improves F1-score by 4%-6%, which demonstrates its effectiveness in multilingual humor de- tection tasks. 1 Introduction Machine learning has been adopted in compu- tational linguistic for understanding natural lan- guages for several decades. With the development of representation learning, rich semantics can be encoded into the dense vectors named as embed- ding, which significantly improves the ability of al- gorithms in understanding fine-grained emotions, for example, judging whether a sentence is hu- morous, often formulated as a binary classification problem. There can be many applications of hu- mor detection such as language understanding in c⃝ 2020 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- BY-ND. Figure 1: An example from HAHA corpus shows that the semantic discrepancy exists in a joke, where the urinated stones is a disease in the left picture and is an action in the right image, originated from the second and the third sentence in the joke: “-Doctor, my kidney hurts a lot -Have you urinated stones? -Yes doctor, I urinated stones, cars, trees, posts ...” dialogue system and sentiment classification in so- cial network platforms. In this paper, we focus on humor detection based on deep learning methods. Many algorithms has been used to solve these problems such as conventional machine learning algorithms like TF-IDF representation with SVM classifier, or deep learning based like BERT (De- vlin et al., 2019). However, most of these algo- rithms are typically designed for universal tasks but ignoring the difference (e.g. the paragraph structure and semantic features) between humor detection and other document classification tasks. From a linguistic perspective, there are two crit- ical features that often appear in jokes, which in- spire us to model them explicitly and make specific optimization for the task: • Good setup and a punchline is the core of many jokes. The setup can be considered as the background of a story, and the punchline is the surprise or the exception that is com- monly contradict to intuition, which is the trigger to make the reader laugh. The punch- line often appears at …

Content truncated to 500 words