Refining Word Embeddings for Sentiment Analysis

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 534–539 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Refining Word Embeddings for Sentiment Analysis Liang-Chih Yu1,3, Jin Wang2,3,4, K. Robert Lai2,3 and Xuejie Zhang4 1Department of Information Management, Yuan Ze University, Taiwan 2Department of Computer Science & Engineering, Yuan Ze University, Taiwan 3Innovation Center for Big Data and Digital Convergence Yuan Ze University, Taiwan 4School of Information Science and Engineering, Yunnan University, Yunnan, P.R. China Contact: lcyu@saturn.yzu.edu.tw Abstract Word embeddings that can capture seman- tic and syntactic information from contexts have been extensively used for various natural language processing tasks. Howev- er, existing methods for learning context- based word embeddings typically fail to capture sufficient sentiment information. This may result in words with similar vec- tor representations having an opposite sen- timent polarity (e.g., good and bad), thus degrading sentiment analysis performance. Therefore, this study proposes a word vec- tor refinement model that can be applied to any pre-trained word vectors (e.g., Word2vec and GloVe). The refinement model is based on adjusting the vector rep- resentations of words such that they can be closer to both semantically and sentimen- tally similar words and further away from sentimentally dissimilar words. Experi- mental results show that the proposed method can improve conventional word embeddings and outperform previously proposed sentiment embeddings for both binary and fine-grained classification on Stanford Sentiment Treebank (SST). 1 Introduction Word embeddings are a technique to learn con- tinuous low-dimensional vector space representa- tions of words by leveraging the contextual in- formation from large corpora. Examples include C&W (Collobert and Weston, 2008; Collobert et al., 2011), Word2vec (Mikolov et al., 2013a; 2013b) and GloVe (Pennington et al., 2014). In addition to the contextual information, character- level subwords (Bojanowski et al., 2016) and se- mantic knowledge resources (Faruqui et al., 2015; Kiela et al., 2015) such as WordNet (Miller, 1995) are also useful information for learning word embeddings. These embeddings have been successfully used for various natural language processing tasks. In general, existing word embeddings are se- mantically oriented. They can capture semantic and syntactic information from unlabeled data in an unsupervised manner but fail to capture suffi- cient sentiment information. This makes it diffi- cult to directly apply existing word embeddings to sentiment analysis. Prior studies have reported that words with similar vector representations (similar contexts) may have opposite sentiment polarities, as in the example of happy-sad men- tioned in (Mohammad et al., 2013) and good-bad in (Tang et al., 2016). Composing these word vec- tors may produce sentence vectors with similar vector representations but opposite sentiment po- larities (e.g., a sentence containing happy and a sentence containing sad may have similar vector representations). Building on such ambiguous vectors will affect sentiment classification per- formance. To enhance the performance of distinguishing words with similar vector representations but op- posite sentiment polarities, recent studies have suggested learning sentiment embeddings from labeled data in a supervised manner (Maas et al., 2011; Labutov and Lipson, 2013; …

Content truncated to 500 words