Thumbs up? Sentiment Classification using Machine Learning Techniques

Thumbs up? Sentiment Classification using Machine Learning Techniques Bo Pang and Lillian Lee Department of Computer Science Cornell University Ithaca, NY 14853 USA {pabo,llee}@cs.cornell.edu Shivakumar Vaithyanathan IBM Almaden Research Center 650 Harry Rd. San Jose, CA 95120 USA shiv@almaden.ibm.com Abstract We consider the problem of classifying doc- uments not by topic, but by overall senti- ment, e.g., determining whether a review is positive or negative. Using movie re- views as data, we find that standard ma- chine learning techniques definitively out- perform human-produced baselines. How- ever, the three machine learning methods we employed (Naive Bayes, maximum en- tropy classification, and support vector ma- chines) do not perform as well on sentiment classification as on traditional topic-based categorization. We conclude by examining factors that make the sentiment classifica- tion problem more challenging. 1 Introduction Today, very large amounts of information are avail- able in on-line documents. As part of the effort to better organize this information for users, researchers have been actively investigating the problem of au- tomatic text categorization. The bulk of such work has focused on topical cat- egorization, attempting to sort documents accord- ing to their subject matter (e.g., sports vs. poli- tics). However, recent years have seen rapid growth in on-line discussion groups and review sites (e.g., the New York Times’ Books web page) where a cru- cial characteristic of the posted articles is their senti- ment, or overall opinion towards the subject matter — for example, whether a product review is pos- itive or negative. Labeling these articles with their sentiment would provide succinct summaries to read- ers; indeed, these labels are part of the appeal and value-add of such sites as www.rottentomatoes.com, which both labels movie reviews that do not con- tain explicit rating indicators and normalizes the different rating schemes that individual reviewers use. Sentiment classification would also be helpful in business intelligence applications (e.g. MindfulEye’s Lexant system1) and recommender systems (e.g., Terveen et al. (1997), Tatemura (2000)), where user input and feedback could be quickly summarized; in- deed, in general, free-form survey responses given in natural language format could be processed using sentiment categorization. Moreover, there are also potential applications to message filtering; for exam- ple, one might be able to use sentiment information to recognize and discard “flames”(Spertus, 1997). In this paper, we examine the effectiveness of ap- plying machine learning techniques to the sentiment classification problem. A challenging aspect of this problem that seems to distinguish it from traditional topic-based classification is that while topics are of- ten identifiable by keywords alone, sentiment can be expressed in a more subtle manner. For example, the sentence “How could anyone sit through this movie?” contains no single word that is obviously negative. (See Section 7 for more examples). Thus, sentiment seems to require more understanding than the usual topic-based classification. So, apart from presenting our results obtained via machine learning techniques, we also analyze the problem to gain a better under- standing of how difficult it is. 2 Previous Work This section briefly …

Content truncated to 500 words