SemEval-2017 Task 5: Fine-Grained Sentiment Analysis on Financial Microblogs and News

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 519–535, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics SemEval-2017 Task 5: Fine-Grained Sentiment Analysis on Financial Microblogs and News Keith Cortis*, Andr´e Freitas*, Tobias Daudert**, Manuela H¨urlimann**, Manel Zarrouk**, Siegfried Handschuh*, and Brian Davis** *Department of Computer Science and Mathematics, University of Passau, Germany **Insight Centre for Data Analytics, National University of Ireland, Galway Abstract This paper discusses the “Fine-Grained Sentiment Analysis on Financial Mi- croblogs and News” task as part of SemEval-2017, specifically under the “Detecting sentiment, humour, and truth” theme. This task contains two tracks, where the first one concerns Microblog messages and the second one covers News Statements and Headlines. The main goal behind both tracks was to predict the sentiment score for each of the mentioned companies/stocks. The sentiment scores for each text instance adopted floating point values in the range of -1 (very negative/bearish) to 1 (very positive/bullish), with 0 designating neutral sentiment. This task attracted a total of 32 participants, with 25 participating in Track 1 and 29 in Track 2. 1 Overview Our task is focused on Sentiment Analysis in the domain of financial microblogs and news. Domain- specific Sentiment Analysis has received much at- tention within the NLP community, motivated by the highly domain-dependent language used to ex- press sentiment (Liu, 2012). Domain-specificity impacts all levels of analysis. On the lexical level, which is crucial in sentiment analysis, Liu (2012) notes that positive words in one domain can be negative in another, and vice versa. For instance, Loughran and McDonald (2011a) show that many words which are considered negative in general- purpose polarity lexicon have a neutral meaning in the financial domain (e.g. “liability”). This makes it difficult to transport sentiment classifiers across do- mains and underlines the need for domain-specific tools. The financial domain is a high-impact use case for Sentiment Analysis because it has been shown that sentiments and opinions can affect market dy- namics (Goonatilake and Herath, 2007; de Kauter et al., 2015). Sentiments are in some cases de- rived from news which discuss macroeconomic fac- tors, company-specific, or political information as all of these can be market-relevant (Sinha, 2014). Good news tends to lift markets and increase opti- mism (de Kauter et al., 2015; Schuster, 2003). Ev- idence has been found that both quantitative mea- sures (e.g. the quantity of news, market fluctu- ation) and qualitative indicators, (e.g. linguistic style and tone) affect investors’ behaviour (Tet- lock et al., 2008; Loughran and McDonald, 2011a; Takala et al., 2014). (Bollen et al., 2011) showed that changes in public mood reflect value shifts in the Dow Jones Industrial Index three to four days later. Given the link between sentiment and market dy- namics, the analysis of public sentiment becomes a powerful method to predict the market reaction. However, the accuracy of machine learning-based sentiment analysis approaches rarely exceeds sev- enty percent (Takala et al., 2014; Eagle Alpha, 2016). Research effort is required to overcome …

Content truncated to 500 words