Assigning Polarity Scores to Reviews Using Machine Learning Techniques

Assigning Polarity Scores to Reviews Using Machine Learning Techniques Daisuke Okanohara1 and Jun’ichi Tsujii1,2,3 1 Department of Computer Science, University of Tokyo, Hongo, 7-3-1, Bunkyo-ku, Tokyo 113-0013 2 CREST, JST, Honcho, 4-1-8, Kawaguchi-shi, Saitama 332-0012 3 School of Informatics, University of Manchester, POBox 88, Sackville St, Manchester, M60 1QD, UK {hillbig, tsujii}@is.s.u-tokyo.ac.jp Abstract. We propose a novel type of document classification task that quantifies how much a given document (review) appreciates the target object using not binary polarity (good or bad) but a continuous mea- sure called sentiment polarity score (sp-score). An sp-score gives a very concise summary of a review and provides more information than binary classification. The difficulty of this task lies in the quantification of po- larity. In this paper we use support vector regression (SVR) to tackle the problem. Experiments on book reviews with five-point scales show that SVR outperforms a multi-class classification method using support vector machines and the results are close to human performance. 1 Introduction In recent years, discussion groups, online shops, and blog systems on the Internet have gained popularity and the number of documents, such as reviews, is growing dramatically. Sentiment classification refers to classifying reviews not by their topics but by the polarity of their sentiment (e.g, positive or negative). It is useful for recommendation systems, fine-grained information retrieval systems, and business applications that collect opinions about a commercial product. Recently, sentiment classification has been actively studied and experimental results have shown that machine learning approaches perform well [13,11,10,20]. We argue, however, that we can estimate the polarity of a review more finely. For example, both reviews A and B in Table 1 would be classified simply as positive in binary classification. Obviously, this classification loses the information about the difference in the degree of polarity apparent in the review text. We propose a novel type of document classification task where we evaluate reviews with scores like five stars. We call this score the sentiment polarity score (sp-score). If, for example, the range of the score is from one to five, we could give five to review A and four to review B. This task, namely, ordered multi-class classification, is considered as an extension of binary sentiment classification. In this paper, we describe a machine learning method for this task. Our system uses support vector regression (SVR) [21] to determine the sp-scores of R. Dale et al. (Eds.): IJCNLP 2005, LNAI 3651, pp. 314–325, 2005. c ⃝ Springer-Verlag Berlin Heidelberg 2005 Assigning Polarity Scores to Reviews 315 Table 1. Examples of book reviews Example of Review binary sp-score (1,...,5) Review A I believe this is very good and a “must read” plus 5 I can’t wait to read the next book in the series. Review B This book is not so bad. plus 4 You may find some interesting points in the book. Table 2. Corpus A: reviews for Harry Potter series book. Corpus B: reviews for all kinds of books. The column of word shows the average number of …

Content truncated to 500 words