Predicting Risk from Financial Reports with Regression

Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the ACL, pages 272–280, Boulder, Colorado, June 2009. c⃝2009 Association for Computational Linguistics Predicting Risk from Financial Reports with Regression Shimon Kogan McCombs School of Business University of Texas at Austin Austin, TX 78712, USA shimon.kogan@mccombs.utexas.edu Dimitry Levin Mellon College of Science Carnegie Mellon University Pittsburgh, PA 15213, USA dimitrylevin@gmail.com Bryan R. Routledge Tepper School of Business Carnegie Mellon University Pittsburgh, PA 15213, USA routledge@cmu.edu Jacob S. Sagi Owen Graduate School of Management Vanderbilt University Nashville, TN 37203, USA Jacob.Sagi@Owen.Vanderbilt.edu Noah A. Smith School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213, USA nasmith@cs.cmu.edu Abstract We address a text regression problem: given a piece of text, predict a real-world continuous quantity associated with the text’s meaning. In this work, the text is an SEC-mandated finan- cial report published annually by a publicly- traded company, and the quantity to be pre- dicted is volatility of stock returns, an empiri- cal measure of financial risk. We apply well- known regression techniques to a large cor- pus of freely available financial reports, con- structing regression models of volatility for the period following a report. Our models ri- val past volatility (a strong baseline) in pre- dicting the target variable, and a single model that uses both can significantly outperform past volatility. Interestingly, our approach is more accurate for reports after the passage of the Sarbanes-Oxley Act of 2002, giving some evidence for the success of that legislation in making financial reports more informative. 1 Introduction We consider a text regression problem: given a piece of text, predict a R-valued quantity associated with that text. Specifically, we use a company’s annual financial report to predict the financial risk of invest- ment in that company, as measured empirically by a quantity known as stock return volatility. Predicting financial risk is of clear interest to anyone who invests money in stocks and central to modern portfolio choice. Financial reports are a government-mandated artifact of the financial world that—one might hypothesize—contain a large amount of information about companies and their value. Indeed, it is an important question whether mandated disclosures are informative, since they are meant to protect investors but are costly to produce. The intrinsic properties of the problem are attrac- tive as a test-bed for NLP research. First, there is no controversy about the usefulness or existential reality of the output variable (volatility). Statisti- cal NLP often deals in the prediction of variables ranging from text categories to linguistic structures to novel utterances. While many of these targets are uncontroversially useful, they often suffer from eval- uation difficulties and disagreement among annota- tors. The output variable in this work is a statistic summarizing facts about the real world; it is not sub- ject to any kind of human expertise, knowledge, or intuition. Hence this prediction task provides a new, objective test-bed for any kind of linguistic analysis. Second, many NLP problems rely on costly anno- tated resources (e.g., treebanks or aligned …

Content truncated to 500 words