Sarcasm Detection in Chinese Using a Crowdsourced Corpus

Sarcasm Detection in Chinese Using a Crowdsourced Corpus 林士凱 Shih-Kai Lin 國立臺灣大學語言學研究所 Graduate Institute of Linguistics National Taiwan University serenity9078@gmail.com 謝舒凱 Shu-Kai Hsieh 國立臺灣大學語言學研究所 Graduate Institute of Linguistics National Taiwan University shukai@gmail.com Abstract Based on the assumption that comment with positive sentimental polarity to a negative issue has high probability to be a sarcasm, we propose a simple yet efficient method to collect sarcastic textual data by crowdsourcing with social media and merging game with a purpose approach. Taking advantage of Facebook's reaction button, posts triggering strong negative emotion are collected. Next, by using PTT's search engine, we successfully connect PTT's comments to the collected posts in Facebook and build the sarcasm corpus. Based on the corpus data, the performance comparison of sarcasm detection between SVM with naïve features and Convolutional Neural Network models is conducted. An impressive accuracy rate and great potentials of the corpus are demonstrated. Keywords: sarcasm, PTT, convolutional neural network, support vector machine, crowdsourcing. 1. Introduction Sentiment analysis is important in automatic interpreting large number of feedbacks from the internet society. However, the usage of sarcasm which typically conveys a negative opinion using positive words could flip the polarity of a message thus interfere the accuracy of the sentiment analysis (Maynard et al. 2014). Therefore, to improve the performance of sentiment analysis model, detection of sarcasm is definitely necessary (Bo et al. 2008). The 2016 Conference on Computational Linguistics and Speech Processing ROCLING 2016, pp. 299-310  The Association for Computational Linguistics and Chinese Language Processing 299 Linguistically, sarcasm has been regarded as a complicated speech act which utters the opposite of what it literally means, and it distinguishes itself with irony in its intention of making the target the butt of derisive contempt (Ling et al. 2016). Sarcasm can be grammaticalized and lexicalized in various patterns, and often requires context-dependent readings with human involvement. Therefore, the construction of sarcasm corpus providing wider windows as well as training data for predictive model has long been considered as an uneasy task. However, with the rapid growth of social media platform like Twitter and Facebook, a new solution is provided via crowdsourcing. For instance, a popular method in previous studies, some groups - al. 2011, Reyes et al. 2012, Liebrecht et al. 2013). Based on the assumption that comment with positive polarity to a negative issue has high probability to be sarcasm, we propose an automatic method to collect sarcastic text data by a two- f k f F k to the comments in Gossiping forum of PTT. Due to the recent progress in machine learning and deep-learning technique, these two method could both handle sarcasm detection as a binary (sarcastic and non-sarcastic utterance) classification problem. However, a performance comparison of sarcasm detection between these two methods has not been conducted before. In this paper, we choose machine learning support vector machine (SVM) and deep-learning convolutional neural network (CNN) to test the difference. Both of them are widely adopted in natural language processing problems (Joachims et al. 1998, …

Content truncated to 500 words