Learning for Microblogs with Distant Supervision: Political Forecasting with Twitter

Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 603–612, Avignon, France, April 23 - 27 2012. c⃝2012 Association for Computational Linguistics Learning for Microblogs with Distant Supervision: Political Forecasting with Twitter Micol Marchetti-Bowick Microsoft Corporation 475 Brannan Street San Francisco, CA 94122 micolmb@microsoft.com Nathanael Chambers Department of Computer Science United States Naval Academy Annapolis, MD 21409 nchamber@usna.edu Abstract Microblogging websites such as Twitter offer a wealth of insight into a popu- lation’s current mood. Automated ap- proaches to identify general sentiment to- ward a particular topic often perform two steps: Topic Identification and Sentiment Analysis. Topic Identification first identi- fies tweets that are relevant to a desired topic (e.g., a politician or event), and Sen- timent Analysis extracts each tweet’s atti- tude toward the topic. Many techniques for Topic Identification simply involve select- ing tweets using a keyword search. Here, we present an approach that instead uses distant supervision to train a classifier on the tweets returned by the search. We show that distant supervision leads to improved performance in the Topic Identification task as well in the downstream Sentiment Anal- ysis stage. We then use a system that incor- porates distant supervision into both stages to analyze the sentiment toward President Obama expressed in a dataset of tweets. Our results better correlate with Gallup’s Presidential Job Approval polls than pre- vious work. Finally, we discover a sur- prising baseline that outperforms previous work without a Topic Identification stage. 1 Introduction Social networks and blogs contain a wealth of data about how the general public views products, campaigns, events, and people. Automated algo- rithms can use this data to provide instant feed- back on what people are saying about a topic. Two challenges in building such algorithms are (1) identifying topic-relevant posts, and (2) iden- tifying the attitude of each post toward the topic. This paper studies distant supervision (Mintz et al., 2009) as a solution to both challenges. We apply our approach to the problem of predicting Presidential Job Approval polls from Twitter data, and we present results that improve on previous work in this area. We also present a novel base- line that performs remarkably well without using topic identification. Topic identification is the task of identifying text that discusses a topic of interest. Most pre- vious work on microblogs uses simple keyword searches to find topic-relevant tweets on the as- sumption that short tweets do not need more so- phisticated processing. For instance, searches for the name “Obama” have been assumed to return a representative set of tweets about the U.S. Pres- ident (O’Connor et al., 2010). One of the main contributions of this paper is to show that keyword search can lead to noisy results, and that the same keywords can instead be used in a distantly super- vised framework to yield improved performance. Distant supervision uses noisy signals in text as positive labels to train classifiers. For in- stance, the token “Obama” can be used to iden- …

Content truncated to 500 words