Crystal: Analyzing Predictive Opinions on the Web

Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 1056–1064, Prague, June 2007. c⃝2007 Association for Computational Linguistics Crystal: Analyzing Predictive Opinions on the Web Soo-Min Kim and Eduard Hovy USC Information Sciences Institute 4676 Admiralty Way, Marina del Rey, CA 90292 {skim,hovy}@ISI.EDU Abstract In this paper, we present an election predic- tion system (Crystal) based on web users’ opinions posted on an election prediction website. Given a prediction message, Crys- tal first identifies which party the message predicts to win and then aggregates predic- tion analysis results of a large amount of opinions to project the election results. We collect past election prediction messages from the Web and automatically build a gold standard. We focus on capturing lexi- cal patterns that people frequently use when they express their predictive opinions about a coming election. To predict elec- tion results, we apply SVM-based super- vised learning. To improve performance, we propose a novel technique which gener- alizes n-gram feature patterns. Experimen- tal results show that Crystal significantly outperforms several baselines as well as a non-generalized n-gram approach. Crystal predicts future elections with 81.68% accu- racy. 1 Introduction As a growing number of people use the Web as a medium for expressing their opinions, the Web is becoming a rich source of various opinions in the form of product reviews, travel advice, social issue discussions, consumer complaints, stock market predictions, real estate market predictions, etc. At least two categories of opinions can be iden- tified. One consists of opinions such as “I like/dislike it”, and the other consists of opinions like “It is likely/unlikely to happen.” We call the first category Judgment Opinions and the second (those discussing the future) Predictive Opinions. Judgment opinions express positive or negative sentiment about a topic such as, for example, re- views about cameras, movies, books, or hotels, and discussions about topics like abortion and war. In contrast, predictive opinions express a person's opinion about the future of a topic or event such as the housing market, a popular sports match, and national election, based on his or her belief and knowledge. Due to the different nature of these two catego- ries of opinion, each has different valences. Judg- ment opinions have core valences of positive and negative. For example, “liking a product” and “supporting abortion” have the valence “positive” toward each topic (namely “a product” and “abor- tion”). Predictive opinions have the core valence of likely or unlikely predicated on the event. For ex- ample, a sentence “Housing prices will go down soon” carries the valence of “likely” for the event of “housing prices go down”. The two types of opinions can co-appear. The sentence “I like Democrats but I think they are not likely to win considering the war issue” contains both types of opinion: “positive” valence towards Democrats and “unlikely” valence towards the event of “Democrats wins”. In order to accurately identify and analyze each type of opinion, different approaches are desirable. …

Content truncated to 500 words