Automatic Domain Adaptation Outperforms Manual Domain Adaptation for Predicting Financial Outcomes

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 346–359 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 346 Automatic Domain Adaptation Outperforms Manual Domain Adaptation for Predicting Financial Outcomes Marina Sedinkina1 Nikolas Breitkopf2 Hinrich Sch¨utze1 1Center for Information & Language Processing, LMU Munich 2Institute for Finance & Banking, LMU Munich sedinkina@cis.uni-muenchen.de Abstract In this paper, we automatically create senti- ment dictionaries for predicting financial out- comes. We compare three approaches: (i) manual adaptation of the domain-general dic- tionary H4N, (ii) automatic adaptation of H4N and (iii) a combination consisting of first man- ual, then automatic adaptation. In our experi- ments, we demonstrate that the automatically adapted sentiment dictionary outperforms the previous state of the art in predicting the finan- cial outcomes excess return and volatility. In particular, automatic adaptation performs bet- ter than manual adaptation. In our analysis, we find that annotation based on an expert’s a priori belief about a word’s meaning can be incorrect – annotation should be performed based on the word’s contexts in the target do- main instead. 1 Introduction Since 1934, the U.S. Securities and Exchange Commission (SEC) mandates that public compa- nies disclose information in form of public fil- ings to ensure that adequate information is avail- able to investors. One such filing is the 10-K, the company’s annual report. It contains financial statements and information about business strat- egy, risk factors and legal issues. For this reason, 10-Ks are an important source of information in the field of finance and accounting. A common method employed by finance and accounting researchers is to evaluate the “tone” of a text based on the Harvard Psychosociologi- cal Dictionary, specifically, on the Harvard-IV-4 TagNeg (H4N) word list.1 However, as its name suggests, this dictionary is from a domain that is different from finance, so many words (e.g., “lia- bility”, “tax”) that are labeled as negative in H4N are in fact not negative in finance. 1http://www.wjh.harvard.edu/˜inquirer In a pioneering study, Loughran and Mcdon- ald (2011) manually reclassified the words in H4N for the financial domain. They applied the result- ing dictionaries2 to 10-Ks and predicted financial variables such as excess return and volatility. We will refer to the sentiment dictionaries created by Loughran and Mcdonald (2011) as L&M. In this work, we also create sentiment dictio- naries for the finance domain, but we adapt them from the domain-general H4N dictionary automat- ically. We first learn word embeddings from a corpus of 10-Ks and then reclassify them – us- ing SVMs trained on H4N labels – as negative vs. non-negative. We refer to the resulting domain- adapted dictionary as H4NRE. In our experiments, we demonstrate that the au- tomatically adapted financial sentiment dictionary H4NRE performs better at predicting excess return and volatility than dictionaries of Loughran and Mcdonald (2011) and Theil et al. (2018). We make the following contributions. (i) We demonstrate that automatic domain adapta- tion performs better at predicting financial out- comes than previous work based on …

Content truncated to 500 words