A Hierarchical Classifier Applied to Multi-way Sentiment Detection

Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 62–70, Beijing, August 2010 A Hierarchical Classifier Applied to Multi-way Sentiment Detection Adrian Bickerstaffe and Ingrid Zukerman Faculty of Information Technology Monash University bickerstaffe.adrian@gmail.com,Ingrid.Zukerman@monash.edu Abstract This paper considers the problem of document-level multi-way sentiment de- tection, proposing a hierarchical classifier algorithm that accounts for the inter-class similarity of tagged sentiment-bearing texts. This type of classifier also pro- vides a natural mechanism for reducing the feature space of the problem. Our re- sults show that this approach improves on state-of-the-art predictive performance for movie reviews with three-star and four- star ratings, while simultaneously reduc- ing training times and memory require- ments. 1 Introduction A key problem in sentiment detection is to deter- mine the polarity of sentiment in text. Much of the work on this problem has considered binary senti- ment polarity (positive or negative) at granularity levels ranging from sentences (Yu and Hatzivas- siloglou, 2003; Mao and Lebanon, 2006; McDon- ald et al., 2007) to documents (Wilson et al., 2005; Allison, 2008). This paper considers the more general problem of multi-way sentiment classification for discrete, ordinal rating scales, focusing on the document level, i.e., the problem of predicting the “star” rat- ing associated with a review. This is a supervised learning task involving textual reviews that have been tagged with a rating. Ultimately, the goal is to use classifiers which have been trained on tagged datasets to predict the ratings of untagged reviews. Typical approaches to the rating scale problem include standard k-way classifiers, e.g., (Pang and Lee, 2005). However, these methods do not ex- plicitly account for sample similarities, e.g., the samples with a “four star” rating being more sim- ilar to “three star” samples than to “one star” sam- ples. Consequently, these methods generally do not perform well, while methods which incor- porate sample similarity information achieve im- proved performance (Pang and Lee, 2005). Sample similarity in the multi-way sentiment detection setting has previously been consid- ered by using Support Vector Machines (SVMs) in conjunction with a metric labeling meta- algorithm (Pang and Lee, 2005); by taking a semi- supervised graph-based learning approach (Gold- berg and Zhu, 2006); and by using “optimal stacks” of SVMs (Koppel and Schler, 2006). However, each of these methods have short- comings (Section 2). Additionally, during the learning process, all approaches employ a set of word/punctuation features collected across all rat- ing categories. Hence, the number of features may be very large compared to the number of training samples, which can lead to the model overfitting the data. The main contribution of this paper is the use of hierarchical classifier trees which combine stan- dard binary classifiers to perform multi-way clas- sification (another approach to reduce multi-class classification to binary classifications is described in (Beygelzimer et al., 2009)). The hierarchi- cal classifier accounts for inter-class similarity by 62 means of tree structures which are obtained using inter-class similarity measures in conjunction with a shortest-spanning algorithm. The tree structures reduce training times …

Content truncated to 500 words