Application and Analysis of a Multi-layered Scheme for Irony on the Italian Twitter Corpus TWITTIRÒ

Application and Analysis of a Multi-layered Scheme for Irony on the Italian Twitter Corpus TWITTIR `O Alessandra Teresa Cignarella1,2, Cristina Bosco1, Viviana Patti1, Mirko Lai1,2 1Dipartimento di Informatica, Universit`a degli Studi di Torino, Italy 2PRHLT Research Center, Universitat Polit`ecnica de Val`encia, Spain {cigna,bosco,patti}@di.unito.it mirko.lai@unito.it Abstract In this paper we describe the main issues emerged within the application of a multi-layered scheme for the fine-grained annotation of irony (Karoui et al., 2017) on an Italian Twitter corpus, i.e. TWITTIR `O, which is composed of about 1,500 tweets with various provenance. A discussion is proposed about the limits and advantages of the application of the scheme to Italian messages, supported by an analysis of the outcome of the annotation carried on by native Italian speakers in the development of the corpus. We present a quantitative and qualitative study both of the distribution of the labels for the different layers involved in the scheme which can shed some light on the process of human annotation for a validation of the annotation scheme on Italian irony-laden social media contents collected in the last years. This results in a novel gold standard for irony detection in Italian, enriched with fine-grained annotations, and in a language resource available to the community and exploitable in the cross- and multi-lingual perspective which characterizes the work that inspired this research. Keywords: irony, figurative language processing, corpora, social media, Italian 1. Introduction The automatic recognition of irony is, still nowadays, a challenging task to be performed both by human annota- tors and automatic NLP systems (Mihalcea and Pulman, 2007; Reyes et al., 2010; Kouloumpis et al., 2011; May- nard and Funk, 2011; Reyes et al., 2012; Hern´andez Far´ıas et al., 2016; Sulis et al., 2016). The growing interest on this task is attested by the proposal of shared tasks focusing on irony detection and its impact on sentiment analysis in social media, in the context of periodical evaluation cam- paigns for NLP tools for many languages, see for instance the pilot task on irony detection proposed for Italian in Sen- tipolc@Evalita, in the 2014 and 2016 editions (Basile et al., 2014; Barbieri et al., 2016) and the battery of related tasks proposed for French at DEFT@TALN2017 (Benamara et al., 2017). For what concerns English, after a first interest- ing task at SemEval-2015 (i.e. Task 11) focusing on Senti- ment Analysis of Figurative Language in Twitter (Ghosh et al., 2015), in 2018 a shared task on irony detection in tweets has been proposed for the first time (SemEval-2018 Task 3: Irony detection in English tweets)1. In the latter, the orga- nizers propose not only the classical binary classification task, where the systems must determine whether a tweet is ironic or not, but also a fine grained multiclass classifi- cation task on different types of irony, where the systems must predict one out of four labels describing: i) verbal irony realized through a polarity contrast, ii) verbal irony without such a polarity contrast, iii) descriptions of situa- tional irony, and iv) non-irony …

Content truncated to 500 words