Annotating Sentiment and Irony in the Online Italian Political Debate on #labuonascuola

Annotating Sentiment and Irony in the Online Italian Political Debate on #labuonascuola Marco Stranisci1,3, Cristina Bosco1, Delia Iraz´u Hern´andez Far´ıas1,2, Viviana Patti1 1Universit`a degli Studi di Torino, Dipartimento di Informatica Corso Svizzera 185 – 10149 Torino, Italy, 2Universitat Polit`ecnica de Val`encia Departamento de Sistemas Inform´aticos y Computaci´on Camino de Vera s/n. 46022 Valencia, Spain 3weLaika, Via Salgari 7, Torino, Italy marco.stranisci@welaika.com, {bosco,patti}@di.unito.it, dhernandez1@dsic.upv.es Abstract In this paper we present the TWitterBuonaScuola corpus (TW-BS), a novel Italian linguistic resource for Sentiment Analysis, developed with the main aim of analyzing the online debate on the controversial Italian political reform “Buona Scuola” (Good school), aimed at reorganizing the national educational and training systems. We describe the methodologies applied in the collection and annotation of data. The collection has been driven by the detection of the hashtags mainly used by the participants to the debate, while the annotation has been focused on sentiment polarity and irony, but also extended to mark the aspects of the reform that were mainly discussed in the debate. An in-depth study of the disagreement among annotators is included. We describe the collection and annotation stages, and the in-depth analysis of disagreement made with Crowdflower, a crowdsourcing annotation platform. Keywords: sentiment analysis, Italian, political debates, Twitter 1. Introduction and Motivation The application of sentiment analysis and opinion mining techniques to the political domain has encountered rising interests in recent years, due to the exponential increase of user-generated contents through social media, such as Twitter, which have been extensively exploited for debat- ing politics and public policies. We can outline two ten- dencies in this area. The first one focuses on studying elec- toral campaigns to gauge the political sentiment (Tumasjan et al., 2011; Sang and Bos, 2012), political trust analysis (Ceron et al., 2014), or users political alignment (Conover et al., 2011). The second one, instead, is less related to po- tential commercial applications, but mainly aimed at inves- tigating socio-political issues (Maynard and Funk, 2011), user’s stance on controversial topics (Rajadesingan and Liu, 2014), or debates on public policies (Gloor et al., 2009; Lai et al., 2015). The present work is in line with the second tendency. We propose a novel data-driven approach to the creation of a manually annotated corpus for Italian sentiment analysis on a popular political debate. Our dataset is composed by Twitter messages about the controversial school reform called La Buona Scuola (Good School), discussed in Italy in 2014. The TWitterBuonaScuola corpus (TW-BS hence- forth) is manually annotated at the message level for senti- ment polarity and irony, in line of previous work on Italian (Bosco et al., 2013; Basile et al., 2014b)1. Moreover, our annotation scheme has been extended in order to mark the ‘aspects‘ of the reform that were mainly discussed in the debate. This opens the way to the exploitation of the corpus 1A preliminary version of project is presented in (Stranisci et al., 2015). for a fine-grained sentiment analysis, where it is possible to go beyond the detection of a generic …

Content truncated to 500 words