The Sentimental Factor: Improving Review Classification Via Human-Provided Information

The Sentimental Factor: Improving Review Classification via Human-Provided Information Philip Beineke∗and Trevor Hastie Dept. of Statistics Stanford University Stanford, CA 94305 Shivakumar Vaithyanathan IBM Almaden Research Center 650 Harry Rd. San Jose, CA 95120-6099 Abstract Sentiment classification is the task of labeling a re- view document according to the polarity of its pre- vailing opinion (favorable or unfavorable). In ap- proaching this problem, a model builder often has three sources of information available: a small col- lection of labeled documents, a large collection of unlabeled documents, and human understanding of language. Ideally, a learning method will utilize all three sources. To accomplish this goal, we general- ize an existing procedure that uses the latter two. We extend this procedure by re-interpreting it as a Naive Bayes model for document sentiment. Viewed as such, it can also be seen to extract a pair of derived features that are linearly combined to predict sentiment. This perspective allows us to improve upon previous methods, primarily through two strategies: incorporating additional derived fea- tures into the model and, where possible, using la- beled data to estimate their relative influence. 1 Introduction Text documents are available in ever-increasing numbers, making automated techniques for infor- mation extraction increasingly useful. Traditionally, most research effort has been directed towards “ob- jective” information, such as classification accord- ing to topic; however, interest is growing in produc- ing information about the opinions that a document contains; for instance, Morinaga et al. (2002). In March, 2004, the American Association for Artifi- cial Intelligence held a symposium in this area, en- titled “Exploring Affect and Attitude in Text.” One task in opinion extraction is to label a re- view document d according to its prevailing senti- ment s ∈ {−1, 1} (unfavorable or favorable). Sev- eral previous papers have addressed this problem by building models that rely exclusively upon la- beled documents, e.g. Pang et al. (2002), Dave et al. (2003). By learning models from labeled data, one can apply familiar, powerful techniques directly; however, in practice it may be difficult to obtain enough labeled reviews to learn model pa- rameters accurately. A contrasting approach (Turney, 2002) relies only upon documents whose labels are unknown. This makes it possible to use a large underlying corpus – in this case, the entire Internet as seen through the AltaVista search engine. As a result, estimates for model parameters are subject to a relatively small amount of random variation. The corresponding drawback to such an approach is that its predictions are not validated on actual documents. In machine learning, it has often been effec- tive to use labeled and unlabeled examples in tan- dem, e.g. Nigam et al. (2000). Turney’s model introduces the further consideration of incorporat- ing human-provided knowledge about language. In this paper we build models that utilize all three sources: labeled documents, unlabeled documents, and human-provided information. The basic concept behind Turney’s model is quite simple. The “sentiment orientation” (Hatzivas- siloglou and McKeown, 1997) of a pair of words is taken to …

Content truncated to 500 words