Sentiment classification on customer feedback data: noisy data, large feature vectors, and the role of linguistic analysis

Sentiment classification on customer feedback data: noisy data, large feature vectors, and the role of linguistic analysis Michael Gamon Microsoft Research One Microsoft Way Redmond, WA 98052 mgamon@microsoft.com Abstract We demonstrate that it is possible to perform automatic sentiment classification in the very noisy domain of customer feedback data. We show that by using large feature vectors in combination with feature reduction, we can train linear support vector machines that achieve high classification accuracy on data that present classification challenges even for a human annotator. We also show that, surprisingly, the addition of deep linguistic analysis features to a set of surface level word n-gram features contributes consistently to classification accuracy in this domain. 1 Introduction Software companies typically receive high volumes of electronic customer feedback every day, some of it in the form of elicited surveys, some of it in the form of unsolicited comments, suggestions, criticism. In order to react to that feedback quickly, and to direct it to the appropriate channels inside the company, it is desirable to provide intelligent and automatic classification of the feedback along two dimensions: What is the feedback about? Is the feedback positive or negative? The first question is addressed by text mining tools. Automatic sentiment classification addresses the second question. Text mining tools can help make large quantities of feedback more manageable by splitting them into clusters based on keywords or topics. Sentiment analysis, which is the focus of this paper, adds a second dimension to the analysis. It makes it possible to focus the text mining on areas in need of improvement (negative feedback) or on areas of success (positive feedback). Sentiment classification is a special case of text categorization, where the criterion of classification is the attitude expressed in the text, rather than the “content” or topic. Faced with the task of having to automatically classify a piece of text as expressing positive or negative sentiment, a reasonable first approach would consist of paying special attention to words that tend to express a positive or negative attitude. Pang et al. (2002) have demonstrated, however, that this is not as straightforward as one may think, given that sentiment is often expressed in more subtle and indirect ways. The literature on sentiment classification can be divided into approaches that rely on semantic resources, such as a sentiment or affect lexicon (Nasukawa and Yi 2003, Subasic and Huettner 2001), or a large scale knowledge base (Liu et al 2003) on the one hand, and approaches that try to learn patterns directly from tagged data, without additional resources (Dave et al 2003, Pang et al. 2003). Much research is also being directed at acquiring affect lexica automatically (Turney 2002, Turney and Littman 2002). There is also a considerable amount of research on classification of text as “subjective” or “objective” (Wiebe et al 2001, Yu and Hatzivassiloglou 2003), a task that is not relevant for the processing of very brief pieces of direct customer feedback. In many studies, research on sentiment classification is conducted …

Content truncated to 500 words