Reducing Over-Weighting in Supervised Term Weighting for Sentiment Analysis

Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 1322–1330, Dublin, Ireland, August 23-29 2014. Reducing Over-Weighting in Supervised Term Weighting for Sentiment Analysis Haibing Wu Department of Electronic Engineering Fudan University Shanghai, China haibingwu13@fudan.edu.cn Xiaodong Gu Department of Electronic Engineering Fudan University Shanghai, China xdgu@fudan.edu.cn Abstract Recently the research on supervised term weighting has attracted growing attention in the field of Tradi- tional Text Categorization (TTC) and Sentiment Analysis (SA). Despite their impressive achievements, we show that existing methods more or less suffer from the problem of over-weighting. Overlooked by prior studies, over-weighting is a new concept proposed in this paper. To address this problem, two regularization techniques, singular term cutting and bias term, are integrated into our framework of su- pervised term weighting schemes. Using the concepts of over-weighting and regularization, we provide new insights into existing methods and present their regularized versions. Moreover, under the guidance of our framework, we develop a novel supervised term weighting scheme, regularized entropy (re). The proposed framework is evaluated on three datasets widely used in SA. The experimental results indicate that our re enjoys the best results in comparisons with existing methods, and regularization techniques can significantly improve the performances of existing supervised weighting methods. 1 Introduction Sentiment Analysis (SA), also known as opinion mining, has enjoyed a burst of research interest with growing avenues (e.g., social networks and e-commerce websites) for people to express their senti- ments on the Internet. A typical sentiment-analysis application mainly involves three key subtasks, namely holder detection, target extraction and sentiment classification (Liu, 2012; Hu and Liu, 2004). A simple and most extensively studied case of sentiment classification is sentiment polarity classifica- tion, which is the binary classification task of labelling the polarity of a sentiment-oriented document as positive or negative. Sentiment classification can be performed at the document, sentence, phase or word level. In this paper, we focus on sentiment polarity classification at document level. Just like Information Retrieval (IR) and TTC, in sentiment classification, the content of an opinion- orientated document can be represented as a vector of terms in light of Vector Space Model (VSM). In VSM, each dimension of the vector corresponds to a term and different terms have different weights, thus the term weight represents the contribution of the term to the sentiment of a document in senti- ment classification. Term weighting is the task of assigning appropriate weights to terms according to their correlations with the category concept. Term weighting schemes fall into two categories (Lan et al., 2009; Debole and Sebastiani, 2003). The first one, known as unsupervised term weighting method, does not take category information into account. The second one referred to as supervised term weighting method embraces the category label information of training documents in the categorization tasks. Although most term weighting approaches to text categorization, including sentiment classifica- tion, are borrowed from IR, recently several new supervised term weighting schemes have been stud- ied and achieved significant successes in TTC …

Content truncated to 500 words