Vote Prediction on Comments in Social Polls

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1127–1138, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Vote Prediction on Comments in Social Polls Isaac Persing and Vincent Ng Human Language Technology Research Institute University of Texas at Dallas Richardson, TX 75083-0688 {persingq,vince}@hlt.utdallas.edu Abstract A poll consists of a question and a set of predefined answers from which voters can select. We present the new problem of vote prediction on comments, which involves determining which of these answers a voter selected given a comment she wrote after voting. To address this task, we ex- ploit not only the information extracted from the comments but also extra-textual information such as user demographic in- formation and inter-comment constraints. In an evaluation involving nearly one mil- lion comments collected from the popu- lar SodaHead social polling website, we show that a vote prediction system that ex- ploits only textual information can be im- proved significantly when extended with extra-textual information. 1 Introduction We introduce in this paper a new opinion mining task, vote prediction on comments in social polls. Recall that a poll consists of a question accompa- nied by a set of predefined answers. A user who votes on the question will choose one of these an- swers and will be prompted to enter a comment giving an explanation of why she chose the an- swer. Given a poll and a user comment written in response to it, the task of vote prediction seeks to determine which predefined answer was chosen by the author of the comment. A solution to the vote prediction problem would contribute significantly to our understanding of the underlying attitudes of individual social polling website users. This understanding could be ex- ploited for tasks such as improving user experi- ence or directed advertising; if we can predict how a user will vote on a question, we can make more accurate guesses about what kind of content/ads related to the question the user would like to see. Unfortunately, a major difficulty of vote predic- tion arises from the casual nature of discussion in social media. A comment often contains insuffi- cient information for inferring the user’s vote, or in some cases may even be entirely absent. In light of this difficulty, we exploit two addi- tional types of information in the prediction pro- cess. First, we employ demographic features de- rived from user profiles. Demographic features may be broadly useful for other opinion mining tasks such as stance classification (Somasundaran and Wiebe, 2010), as many social media web- sites like CreateDebate1 allow users to create pro- files with similar demographic information. Previ- ous work has attempted to predict such latent fea- tures (e.g., Rao and Yarowsky (2010), Burger et al. (2011)) rather than employing them for opin- ion mining tasks. Second, we exploit inter-comment constraints to help us perform joint inference over votes on different questions. Note that previous work on debate stance recognition has also employed con- straints to improve …

Content truncated to 500 words