UW-FinSent at SemEval-2017 Task 5: Sentiment Analysis on Financial News Headlines using Training Dataset Augmentation

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 872–876, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics UW-FinSent at SemEval-2017 Task 5: Sentiment Analysis on Financial News Headlines using Training Dataset Augmentation Vineet John University of Waterloo vineet.john@uwaterloo.ca Olga Vechtomova University of Waterloo ovechtomova@uwaterloo.ca Abstract This paper discusses the approach taken by the UWaterloo team to arrive at a solu- tion for the Fine-Grained Sentiment Anal- ysis problem posed by Task 5 of SemEval 2017. The paper describes the document vectorization and sentiment score predic- tion techniques used, as well as the de- sign and implementation decisions taken while building the system for this task. The system uses text vectorization mod- els, such as N-gram, TF-IDF and para- graph embeddings, coupled with regres- sion model variants to predict the senti- ment scores. Amongst the methods exam- ined, unigrams and bigrams coupled with simple linear regression obtained the best baseline accuracy. The paper also explores data augmentation methods to supplement the training dataset. This system was de- signed for Subtask 2 (News Statements and Headlines). 1 Introduction The goal of this SemEval task is to identify fine- grained levels of sentiment polarity in financial news headlines and microblog posts. Specifically, the task aims at identifying bullish (optimistic) sentiment, expressing the belief that the stock price will increase, and bearish (pessimistic) sen- timent, expressing the belief that the stock price will decline. The expressed sentiment is quan- tified as floating point values in the range of -1 (very negative/bearish) to 1 (very positive/bullish), with 0 denoting neutral sentiment. (Cortis et al., 2017). This paper describes our system developed for subtask 2 (News Statements and Headlines). While developing the system for this subtask, we systematically evaluated a number of alterna- tive solutions for each step in the pipeline. Specif- ically, we investigated different document vector- ization approaches, such as N-gram models, TF- IDF and paragraph vectors. A number of regres- sion models were evaluated, namely, Simple Lin- ear Regression, Support Vector Regression and XGBoost Linear Regression. One of the challenges with performing senti- ment analysis in the financial domain is scarcity of training data. We explored different approaches to augment the training data provided by the task organizers with training data from other sources in the financial domain, as well as using out-of- domain sentiment resources. 2 Approach The overall approach to predicting the sentiment of the test dataset headlines is detailed below. • Pre-Processing & Cleaning This step is needed to simplify and sanitize the input set of headlines. In the context of this task, since the headlines were short snip- pets ranging from 5 to 15 words in length, the only pre-processing done was replacing the name of the organization being spoken of in the headlines, with a generic organization name, to reduce the feature space. • Text Vectorization The objective is to vectorize the textual con- tent of the headlines into a numeric repre- sentation that a statistical learning model can then be …

Content truncated to 500 words