Experimental Evaluation of a Lexicon- and Corpus-based Ensemble for Multi-way Sentiment Analysis

Experimental Evaluation of a Lexicon- and Corpus-based Ensemble for Multi-way Sentiment Analysis∗ Minh Duc Cao School of Chemistry and Molecular Biosciences The University of Queensland St Lucia, QLD 4072, Australia m.cao1@uq.edu.au Ingrid Zukerman Clayton School of Information Technology Monash University Clayton, VIC 3800, Australia Ingrid.Zukerman@monash.edu Abstract We describe a probabilistic approach that combines information obtained from a lexi- con with information obtained from a Na¨ıve Bayes (NB) classifier for multi-way sen- timent analysis. Our approach also em- ploys grammatical structures to perform ad- justments for negations, modifiers and sen- tence connectives. The performance of this method is compared with that of an NB classifier with feature selection, and MCST – a state-of-the-art system. The results of our evaluation show that the performance of our hybrid approach is at least as good as that of these systems. We also exam- ine the influence of three factors on per- formance: (1) sentiment-ambiguous sen- tences, (2) probability of the most proba- ble star rating, and (3) coverage of the lexi- con and the NB classifier. Our results indi- cate that the consideration of these factors supports the identification of regions of im- proved reliability for sentiment analysis. 1 Introduction A key problem in sentiment analysis is to deter- mine the polarity of sentiment in text. Much of the work on this problem has considered binary sentiment polarity (positive or negative) at gran- ularity levels ranging from sentences (Mao and Lebanon, 2006; McDonald et al., 2007) to docu- ments (Wilson et al., 2005; Allison, 2008). Multi- way polarity classification, i.e., the problem of inferring the “star” rating associated with a re- view, has been attempted in several domains, e.g., restaurant reviews (Snyder and Barzilay, 2007) *The majority of this work was done while the first author was at Monash University. and movie reviews (Bickerstaffe and Zukerman, 2010; Pang and Lee, 2005). Star ratings are more informative than positive/negative ratings, and are commonly given in reviews of films, restaurants, books and consumer goods. However, because of this finer grain, multi-way sentiment classification is a more difficult task than binary classification. Hence, the results for multi-way classification are typically inferior to those obtained for the binary case. Most of the research on sentiment analysis uses supervised classification methods such as Maxi- mum Entropy (Berger et al., 1996), Support Vec- tor Machines (SVMs) (Cortes and Vapnik, 1995) or Na¨ıve Bayes (NB) (Domingos and Pazzani, 1997). The sentiment expressed in word pat- terns has been exploited by considering word n- grams (Hu et al., 2007), applying feature selec- tion to handle the resultant proliferation of fea- tures (Mukras et al., 2007). In addition, when per- forming multi-way classification, approaches that consider class-label similarities (Bickerstaffe and Zukerman, 2010; Pang and Lee, 2005) generally outperform those that do not. Lexicon-based methods for sentiment analy- sis have been investigated in (Beineke et al., 2004; Taboada et al., 2011; Andreevskaia and Bergler, 2008; Melville et al., 2009) in the con- text of binary, rather than multi-way, sentiment classifiers. These methods often require …

Content truncated to 500 words