Bootstrap Domain-Specific Sentiment Classifiers from Unlabeled Corpora

Bootstrap Domain-Specific Sentiment Classifiers from Unlabeled Corpora Andrius Mudinas, Dell Zhang, and Mark Levene Department of Computer Science and Information Systems Birkbeck, University of London London WC1E 7HX, UK andrius@dcs.bbk.ac.uk, dell.z@ieee.org, mark@dcs.bbk.ac.uk Abstract There is often the need to perform sentiment classification in a particular domain where no labeled document is available. Although we could make use of a general-purpose off-the- shelf sentiment classifier or a pre-built one for a different domain, the effectiveness would be inferior. In this paper, we explore the possibil- ity of building domain-specific sentiment clas- sifiers w ith u nlabeled d ocuments o nly. Our investigation indicates that in the word em- beddings learned from the unlabeled corpus of a given domain, the distributed word rep- resentations (vectors) for opposite sentiments form distinct clusters, though those clusters are not transferable across domains. Ex- ploiting such a clustering structure, we are able to utilize machine learning algorithms to induce a quality domain-specific senti- ment lexicon from just a few typical senti- ment words (“seeds”). An important finding is that simple linear model based supervised learning algorithms (such as linear SVM) can actually work better than more sophis- ticated semi-supervised/transductive learning algorithms which represent the state-of-the- art technique for sentiment lexicon induction. The induced lexicon could be applied directly in a lexicon-based method for sentiment clas- sification, but a higher performance could be achieved through a two-phase bootstrapping method which uses the induced lexicon to as- sign positive/negative sentiment scores to un- labeled documents first, a nd t hen u ses those documents found to have clear sentiment sig- nals as pseudo-labeled examples to train a document sentiment classifier via supervised learning algorithms (such as LSTM). On sev- eral benchmark datasets for document senti- ment classification, our end-to-end pipelined approach which is overall unsupervised (ex- cept for a tiny set of seed words) outper- forms existing unsupervised approaches and achieves an accuracy comparable to that of fully supervised approaches. 1 Introduction Sentiment analysis (Liu, 2015) is a popular research topic which has a wide range of applications, such as summarizing customer reviews, monitoring social media, and predicting stock market trends (Bollen et al., 2011). A basic task in sentiment analysis is to classify the sentiment polarity of a given piece of text (document), i.e., whether the opinion expressed in the text is positive or negative (Pang et al., 2002), which is the focus of this paper. There are many different approaches to senti- ment classification in the Natural Language Process- ing (NLP) literature — from simple lexicon-based methods (Ding et al., 2008; Thelwall et al., 2010; Thelwall et al., 2012) to learning-based approaches (Pang and Lee, 2004; Turney, 2002; Jo and Oh, 2011; Argamon et al., 2007; Lin and He, 2009), and also hybrid methods in between (Mudinas et al., 2012; Zhang et al., 2011). No matter which ap- proach is taken, a sentiment classifier built for its target domain would work well only within that spe- cific domain, but suffer a serious performance loss once the …

Content truncated to 500 words