Improvement of a Naive Bayes Sentiment Classifier Using MRS-Based Features

Proceedings of the Third Joint Conference on Lexical and Computational Semantics (*SEM 2014), pages 22–29, Dublin, Ireland, August 23-24 2014. Improvement of a Naive Bayes Sentiment Classifier Using MRS-Based Features Jared Kramer University of Washington Seattle, WA jaredkk@uw.edu Clara Gordon University of Washington Seattle, WA cgordon1@uw.edu Abstract This study explores the potential of us- ing deep semantic features to improve bi- nary sentiment classification of paragraph- length movie reviews from the IMBD website. Using a Naive Bayes classifier as a baseline, we show that features extracted from Minimal Recursion Semantics repre- sentations in conjunction with back-off re- placement of sentiment terms is effective in obtaining moderate increases in accu- racy over the baseline’s n-gram features. Although our results are mixed, our most successful feature combination achieves an accuracy of 89.09%, which represents an increase of 0.76% over the baseline per- formance and a 6.48% reduction in error. 1 Introduction Text-based sentiment analysis offers valuable in- sight into the opinions of large communities of re- viewers, commenters and customers. In their sur- vey of the field, Pang and Lee (2008) highlight the importance of sentiment analysis across a range of industries, including review aggregation web- sites, business intelligence, and reputation man- agement. Detection and classification of sentiment can improve downstream performance in applica- tions sensitive to user opinions, such as question- answering, automatic product recommendations, and social network analysis (ibid., p. 12). While previous research in sentiment analysis has investigated the extraction of features from syntactic dependency trees, semantic representa- tions appear to be underused as a resource for modeling opinion in text. Indeed, to our knowl- edge, there has been no research using seman- tic dependencies created by a precision grammar for sentiment analysis. The goal of the present research is to address this gap by augmenting a baseline classifier with features based on Min- imal Resursion Semantics (MRS; Copestake et al., 2005), a formal semantic representation pro- vided by the English Resource Grammar (ERG; Flickinger, 2000). An MRS is a connected graph in which semantic entities may be linked directly through shared arguments or indirectly through handle or qeq constraints, which denote equal- ity modulo quantifier insertion (Copestake et al., 2005). This schema allows for underspecification of quantifier scope. Using Narayanan et al.’s (2013) Naive Bayes sentiment classifier as a baseline, we test the effec- tiveness of eight feature types derived from MRS. Our feature pipeline crawls various links in the MRS representations of sentences in our corpus of paragraph-length movie reviews and outputs simple, human-readable features based on various types of semantic relationships. This improved system achieves modest increases in binary senti- ment classification accuracy for several of the fea- ture combinations tested.1 In the following sections, we summarize previ- ous research in MRS feature extraction and senti- ment classification, describe the baseline system and our modifications to it, and outline our ap- proach to parsing our data, constructing features, and integrating them into the existing system. Fi- nally, we report our findings, examine in more detail where …

Content truncated to 500 words