Financial Sentiment Analysis: An Investigation into Common Mistakes and Silver Bullets

Proceedings of the 28th International Conference on Computational Linguistics, pages 978–987 Barcelona, Spain (Online), December 8-13, 2020 978 Financial Sentiment Analysis: An Investigation into Common Mistakes and Silver Bullets Frank Z. Xing†, Lorenzo Malandri‡, Yue Zhang∗, Erik Cambria† †Nanyang Technological University, Singapore ‡University of Milano-Bicocca, Italy ∗Westlake University, China {zhutian.xing, cambria}@ntu.edu.sg lorenzo.malandri@unimib.it, zhangyue@westlake.edu.cn Abstract The recent dominance of machine learning-based natural language processing methods has fos- tered the culture of overemphasizing model accuracies rather than studying the reasons behind their errors. Interpretability, however, is a critical requirement for many downstream AI and NLP applications, e.g., in finance, healthcare, and autonomous driving. This study, instead of propos- ing any “new model”, investigates the error patterns of some widely acknowledged sentiment analysis methods in the finance domain. We discover that (1) those methods belonging to the same clusters are prone to similar error patterns, and (2) there are six types of linguistic features that are pervasive in the common errors. These findings provide important clues and practical considerations for improving sentiment analysis models for financial applications. 1 Introduction Natural language processing has been widely used for financial applications in recent years. These appli- cations include stock market / foreign exchange market prediction, volatility modeling, asset allocation, business taxonomy construction, credit scoring, initial public offering valuation (IPO), and more (Ding et al., 2015; Xing et al., 2018a; Bai et al., 2019; Xing et al., 2019; Yang et al., 2020; Xing et al., 2021). Among them, there are mainly two threads of method to process textual inputs: the first is to directly encode financial texts by neural nets and to use the representations for learning downstream tasks (Xu and Cohen, 2018); the second is to analyze financial texts with critical linguistic features such as con- tent semantics (Keith and Stent, 2019) or investors’ sentiment (Malandri et al., 2018), for the sake of interpretability. The objective of financial sentiment analysis (FSA) is to classify a piece of financial text as expressing bullish or bearish opinions toward certain arguments. Although sentiment analysis in the general domain is extensively studied in the past decades (Cambria et al., 2013), FSA is a challenging task because of the lack of large-scale training data and the difficulty in labeling after acquiring the texts, which involves expert knowledge. As a result, the model performance for FSA are usually significantly worse than using the same sentiment analysis model for the general domain (which is referred to as a problem of domain adaptation). In addition to the above-mentioned challenges, FSA requires more interpretability comparing to senti- ment analysis in other domains (Luo et al., 2018; Sedinkina et al., 2019). Early sentiment analysis studies leveraged on textual data from movie reviews, product reviews, and social media posts. In these appli- cations, the purpose being roughly understanding customer feedbacks, statistically aggregating straight- forward opinions suffices and a single mistake does not make much difference. Whereas for financial applications, a fraction of bad sentiment analysis results may cause extreme loss, hence have to be care- fully …

Content truncated to 500 words