“i have a feeling trump will win..................”: Forecasting Winners and Losers from User Predictions on Twitter

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1583–1592 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics “i have a feeling trump will win..................”: Forecasting Winners and Losers from User Predictions on Twitter Sandesh Swamy, Alan Ritter Computer Science & Engineering The Ohio State University Columbus, OH swamy.14@osu.edu, aritter@cse.ohio-state.edu Marie-Catherine de Marneffe Department of Linguistics The Ohio State University Columbus, OH mcdm@ling.ohio-state.edu Abstract Social media users often make explicit predictions about upcoming events. Such statements vary in the degree of certainty the author expresses toward the outcome: “Leonardo DiCaprio will win Best Actor” vs. “Leonardo DiCaprio may win” or “No way Leonardo wins!”. Can popular be- liefs on social media predict who will win? To answer this question, we build a cor- pus of tweets annotated for veridicality on which we train a log-linear classifier that detects positive veridicality with high pre- cision.1 We then forecast uncertain out- comes using the wisdom of crowds, by ag- gregating users’ explicit predictions. Our method for forecasting winners is fully automated, relying only on a set of con- tenders as input. It requires no training data of past outcomes and outperforms sentiment and tweet volume baselines on a broad range of contest prediction tasks. We further demonstrate how our approach can be used to measure the reliability of individual accounts’ predictions and retro- spectively identify surprise outcomes. 1 Introduction In the digital era we live in, millions of peo- ple broadcast their thoughts and opinions online. These include predictions about upcoming events of yet unknown outcomes, such as the Oscars or election results. Such statements vary in the ex- tent to which their authors intend to convey the event will happen. For instance, (a) in Table 1 strongly asserts the win of Natalie Portman over Meryl Streep, whereas (b) imbues the claim with 1The code and data can be found at https://github. com/SandeshS/Twitter-Veridicality (a) Natalie Portman is gonna beat out Meryl Streep for best actress (b) La La Land doesn’t have lead actress and actor guaranteed. Natalie Portman will probably (and should) get best actress (c) Adored #LALALAND but it’s #NataliePortman who deserves the best actress #oscar #OscarNoms > superb acting Table 1: Examples of tweets expressing vary- ing degrees of veridicality toward Natalie Portman winning an Oscar. uncertainty. In contrast, (c) does not say anything about the likelihood of Natalie Portman winning (although it clearly indicates the author would like her to win). Prior work has made predictions about contests such as NFL games (Sinha et al., 2013) and elec- tions using tweet volumes (Tumasjan et al., 2010) or sentiment analysis (O’Connor et al., 2010; Shi et al., 2012). Many such indirect signals have been shown useful for prediction, however their utility varies across domains. In this paper we explore whether the “wisdom of crowds” (Surowiecki, 2005), as measured by users’ explicit predictions, can predict outcomes of future events. We show how it is possible to accurately forecast winners, by aggregating many individual predictions that …

Content truncated to 500 words