Ciron: a New Benchmark Dataset for Chinese Irony Detection

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 5714–5720 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 5714 Ciron: a New Benchmark Dataset for Chinese Irony Detection Rong Xiang1, Xuefeng Gao2, Yunfei Long34, Anran Li2, Emmanuele Chersoni2, Qin Lu1, Chu-Ren Huang2 The Hong Kong Polytechnic University12, University of Nottingham3, University of Essex4 Department of Computing, 11 Yuk Choi Road, Hung Hom, Hong Kong (China)1 Chinese and Bilingual Studies, 11 Yuk Choi Road, Hung Hom, Hong Kong (China)2 NIHR Nottingham Biomedical Research Centre, Nottingham (UK) 3 School of Computer Science and Electronic Engineering, University of Essex, Colchester (UK)4 xiangrong0302@gmail.com1, csluqin@comp.polyu.edu.hk1, {xue-feng.gao,an-ran.li}@connect.polyu.hk2, yunfei.long@nottingham.ac.uk3, {emmanuele.chersoni,churen.huang}@polyu.edu.hk2 Abstract Automatic Chinese irony detection is a challenging task, and it has a strong impact on linguistic research. However, Chinese irony detection often lacks labeled benchmark datasets. In this paper, we introduce Ciron, the first Chinese benchmark dataset available for irony detection for machine learning models. Ciron includes more than 8.7K posts, collected from Weibo, a micro blogging platform. Most importantly, Ciron is collected with no pre-conditions to ensure a much wider coverage. Evaluation on seven different machine learning classifiers proves the usefulness of Ciron as an important resource for Chinese irony detection. Keywords: Irony detection, Chinese benchmark dataset, social media text, text processing 1. Introduction The development of the social web has stimulated the use of figurative and creative language in public in- cluding the use of irony. As a special kind of figurative device, the most striking feature of irony is the incon- gruity between the literal meaning and the contextual meaning of an ironic sentence (Farías et al., 2016). Al- though a unanimous definition of irony is still lacking in the literature, it is often identified as a trope whose actual meaning differs from what is literally enunci- ated. Due to its nature, irony detection is very important for natural language processing (NLP) tasks, especially if tasks aim at automatically understanding human lan- guages. Indeed, automatic irony detection has a large potential for various applications in the domain of text mining, especially those that require semantic analysis, such as author profiling, online harassment and hate speech detection, and perhaps, the most well-known task of affective analysis. Compared to other text analysis tasks, irony detection has received limited computational treatment (Barbi- eri and Saggion, 2014; Ghosh et al., 2019). Affective analysis works started to analyze and summarize lin- guistic features of irony from a sentiment shifting per- spective in order to allow for its computational for- malization (Ebert et al., 2015; Long et al., 2019). While theories based on English have provided a rel- atively comprehensive map of irony, there is still a lack of literature dealing with non-Indo-European lan- guages. Even though irony is a pervasive linguistic phenomenon, some of its features vary in different cul- tures and in structural properties of a specific language (Xing and Xu, 2015). For example, Karoui and col- leagues (2019) suggested capitalized words as a strong hint of irony …

Content truncated to 500 words