The SSIX Corpora: Three Gold Standard Corpora for Sentiment Analysis in English, Spanish and German Financial Microblogs

The SSIX Corpora: Three Gold Standard Corpora for Sentiment Analysis in English, Spanish and German Financial Microblogs Thomas Gaillat, Manel Zarrouk, Andr´e Freitas, Brian Davis Insight Centre for Data Analytics NUIG, Insight Centre for Data Analytics NUIG, University of Passau, Maynooth University Galway Ireland, Galway Ireland, Passau Germany, Maynooth Ireland {thomas.gaillat, manel.zarrouk}@insight-centre.org andre.freitas@uni-passau.de brian.davis@mu.ie Abstract This paper introduces the three SSIX corpora for sentiment analysis. These corpora address the need to provide annotated data for supervised learning methods. They focus on stock-market related messages extracted from two financial microblog platforms, i.e., StockTwits and Twitter. In total they include 2,886 messages with opinion targets. These messages are provided with polarity annotation set on a continuous scale by three or four experts in each language. The annotation information identifies the targets with a sentiment score. The annotation process includes manual annotation verified and consolidated by financial experts. The creation of the annotated corpora took into account principled sampling strategies as well as inter-annotator agreement before consolidation in order to maximize data quality. Keywords: Sentiment Analysis, Opinion, Corpus, Finance, Stock-market, Microblogs, Polarity Annotation 1. Introduction The objective of this paper is to report on the creation of three Gold Standard (GS) corpora in the domain of Senti- ment Analysis (SA) related to financial microblogs. The purpose is to create three annotated corpora which support the quantification of the polarity of an opinion with regard to target named entities (e.g. a stock or a company). The produced gold standards represent sentiment values in a fine-grained fashion: (i) by identifying the polarity in re- lation to specific target entities and (ii) by using a continu- ous polarity scale. The corpora are in English, German and Spanish. Today’s ubiquity of social media services in people’s lives has lead to the daily posting of vast amounts of data from users in many languages. Many of these postings bear a common characteristic that is of holding judgment. Com- piling opinions has become a major stake for organisations and people seeking to have a clear view of opinions regard- ing specific entities of interest. In a highly competitive and volatile domain such as finance, acquiring an accurate vi- sion of operators’ opinions at scale may give a competitive edge in sell-and-buy decisions. Microblogs posted on so- cial media such as Twitter 1 or StockTwits2 are a central data source to understand the market perception. Research in the area of Sentiment Analysis (SA) can help address this need but it faces two kinds of issues. Firstly, SA is domain dependent. Much effort has been spent in the field to assign sentiments to various types of texts. Ini- tially focused on product reviews the research has moved to other domains and other types of texts such as microblogs. SA results show that different topics yield differences in SA accuracy (Liu, 2012, p. 36). While SA models for product reviews may already have achieved high classification ac- curacy, other types of texts might still present specific lin- 1https://twitter.com/ 2https://stocktwits.com/ guistic analysis and …

Content truncated to 500 words