Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification

Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 1555–1565, Baltimore, Maryland, USA, June 23-25 2014. c⃝2014 Association for Computational Linguistics Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification∗ Duyu Tang†, Furu Wei‡ , Nan Yang♮, Ming Zhou‡, Ting Liu†, Bing Qin† †Research Center for Social Computing and Information Retrieval Harbin Institute of Technology, China ‡Microsoft Research, Beijing, China ♮University of Science and Technology of China, Hefei, China {dytang, tliu, qinb}@ir.hit.edu.cn {fuwei, v-nayang, mingzhou}@microsoft.com Abstract We present a method that learns word em- bedding for Twitter sentiment classifica- tion in this paper. Most existing algorithm- s for learning continuous word represen- tations typically only model the syntactic context of words but ignore the sentimen- t of text. This is problematic for senti- ment analysis as they usually map word- s with similar syntactic context but oppo- site sentiment polarity, such as good and bad, to neighboring word vectors. We address this issue by learning sentiment- specific word embedding (SSWE), which encodes sentiment information in the con- tinuous representation of words. Specif- ically, we develop three neural networks to effectively incorporate the supervision from sentiment polarity of text (e.g. sen- tences or tweets) in their loss function- s. To obtain large scale training corpora, we learn the sentiment-specific word em- bedding from massive distant-supervised tweets collected by positive and negative emoticons. Experiments on applying SS- WE to a benchmark Twitter sentimen- t classification dataset in SemEval 2013 show that (1) the SSWE feature performs comparably with hand-crafted features in the top-performed system; (2) the perfor- mance is further improved by concatenat- ing SSWE with existing feature set. 1 Introduction Twitter sentiment classification has attracted in- creasing research interest in recent years (Jiang et al., 2011; Hu et al., 2013). The objective is to clas- sify the sentiment polarity of a tweet as positive, ∗ This work was done when the first and third authors were visiting Microsoft Research Asia. negative or neutral. The majority of existing ap- proaches follow Pang et al. (2002) and employ ma- chine learning algorithms to build classifiers from tweets with manually annotated sentiment polar- ity. Under this direction, most studies focus on designing effective features to obtain better clas- sification performance. For example, Mohammad et al. (2013) build the top-performed system in the Twitter sentiment classification track of SemEval 2013 (Nakov et al., 2013), using diverse sentiment lexicons and a variety of hand-crafted features. Feature engineering is important but labor- intensive. It is therefore desirable to discover ex- planatory factors from the data and make the learn- ing algorithms less dependent on extensive fea- ture engineering (Bengio, 2013). For the task of sentiment classification, an effective feature learn- ing method is to compose the representation of a sentence (or document) from the representation- s of the words or phrases it contains (Socher et al., 2013b; Yessenalina and Cardie, 2011). Ac- cordingly, it is a crucial step to learn the word representation (or word embedding), which is a dense, low-dimensional and real-valued vector …

Content truncated to 500 words