A Sentiment Detection Engine for Internet Stock Message Boards

A Sentiment Detection Engine for Internet Stock Message Boards Christopher C. Chua †‡ Maria Milosavljevic ‡ James R. Curran ‡∗ School of Computer Science Capital Markets CRC Ltd ‡ School of Information and Engineering† Technologies ∗ University of New South Wales 55 Harrington Street University of Sydney NSW 2052, Australia NSW 2000, Australia NSW 2006, Australia {cchua,maria}@cmcrc.com james@it.usyd.edu.au Abstract Financial investors trade on the basis of infor- mation, and in particular, on the likelihood that a piece of information will impact the mar- ket. The ability to predict this within millisec- onds of the information being released would be useful in applications such as algorithmic trading. We present a solution for classifying investor sentiment on internet stock message boards. Our solution develops upon prior work and examines several approaches for selecting features in a messy and sparse data set. Us- ing a variation of the Bayes classifier with fea- ture selection methods allows us to produce a system with better accuracy, execution perfor- mance and precision than using conventional Na¨ıve Bayes and SVM classifiers. Evaluation against author-selected sentiment labels results in an accuracy of 78.72% compared to a hu- man annotation and conventional Na¨ıve Bayes accuracy of 57% and 65.63% respectively. 1 Introduction In this paper, we present a sentiment prediction en- gine for classifying investor sentiment, i.e. sig- nals to buy, sell or hold stock positions, based on messages posted on internet stock forums. Our sentiment annotated corpus comes from HotCop- per1, the most popular investment forum for the Australian market, where posts include author self- reported sentiment labels. This unique character- istic of this data set present us with an opportu- nity to extend research in sentiment classification. 1http://www.hotcopper.com.au Our automated sentiment detection engine imple- mentation uses variations classifiers, particularly the Bernoulli Na¨ıve Bayes and the Complement Na¨ıve Bayes (CNB) models, coupled with feature selection techniques using InfoGain, phrase polarity counts and price alerts. Our methods achieve 78.72% ac- curacy for CNB and 78.45% for Bernoulli. These figures are higher than the 57% accuracy from hu- man annotators and 65.63% in the baseline. It also outperforms results from Das and Chen (2007) on a different dataset. 2 Problem Domain Our results contribute towards the development of a real-time solution which monitors financial informa- tion in order to provide useful advice to support mar- ket surveillance analysts’ task of explaining alerts surrounding price movements in stocks. For exam- ple, when the overall sentiment towards a particular stock is positive, it may well explain the observed increase in its uptake. Many forums do not pro- vide an ability for authors to explicitly report sen- timent, thus we hope to eventually apply this model to other forums. The “Buy”, “Hold” and “Sell” tags are analogous to positive, neutral and negative senti- ments respectively. HotCopper also includes a finer- grained labeling system for “short term” and “long term” sentiments. However such distinctions are be- yond our current scope because a finer granularity in the recommendation strength given limited contex- tual information is …

Content truncated to 500 words