Reading the Markets: Forecasting Public Opinion of Political Candidates by News Analysis

Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 473–480 Manchester, August 2008 Reading the Markets: Forecasting Public Opinion of Political Candidates by News Analysis Kevin Lerman Dept. of Computer Science Columbia University New York, NY USA klerman@cs.columbia.edu Ari Gilder and Mark Dredze Dept. of CIS University of Pennsylvania Philadelphia, PA USA agilder@alumni.upenn.edu mdredze@cis.upenn.edu Fernando Pereira Google, Inc. 1600 Amphitheatre Parkway Mountain View, CA USA pereira@google.com Abstract Media reporting shapes public opinion which can in turn influence events, partic- ularly in political elections, in which can- didates both respond to and shape public perception of their campaigns. We use computational linguistics to automatically predict the impact of news on public per- ception of political candidates. Our sys- tem uses daily newspaper articles to pre- dict shifts in public opinion as reflected in prediction markets. We discuss various types of features designed for this problem. The news system improves market predic- tion over baseline market systems. 1 Introduction The mass media can affect world events by sway- ing public opinion, officials and decision makers. Financial investors who evaluate the economic per- formance of a company can be swayed by positive and negative perceptions about the company in the media, directly impacting its economic position. The same is true of politics, where a candidate’s performance is impacted by media influenced pub- lic perception. Computational linguistics can dis- cover such signals in the news. For example, De- vitt and Ahmad (2007) gave a computable metric of polarity in financial news text consistent with human judgments. Koppel and Shtrimberg (2004) used a daily news analysis to predict financial mar- ket performance, though predictions could not be used for future investment decisions. Recently, c⃝ 2008. Licensed under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported li- cense (http://creativecommons.org/licenses/by-nc-sa/3.0/). Some rights reserved. a study conducted of the 2007 French presiden- tial election showed a correlation between the fre- quency of a candidate’s name in the news and elec- toral success (V´eronis, 2007). This work forecasts day-to-day changes in pub- lic perception of political candidates from daily news. Measuring daily public perception with polls is problematic since they are conducted by a variety of organizations at different intervals and are not easily comparable. Instead, we rely on daily measurements from prediction markets. We present a computational system that uses both external linguistic information and internal market indicators to forecast public opinion mea- sured by prediction markets. We use features from syntactic dependency parses of the news and a user-defined set of market entities. Successive news days are compared to determine the novel component of each day’s news resulting in features for a machine learning system. A combination sys- tem uses this information as well as predictions from internal market forces to model prediction markets better than several baselines. Results show that news articles can be mined to predict changes in public opinion. Opinion forecasting differs from that of opin- ion analysis, such as extracting opinions, evaluat- ing sentiment, and extracting predictions (Kim and Hovy, 2007). …

Content truncated to 500 words