Encoding Sentiment Information into Word Vectors for Sentiment Analysis

Proceedings of the 27th International Conference on Computational Linguistics, pages 997–1007 Santa Fe, New Mexico, USA, August 20-26, 2018. 997 Encoding Sentiment Information into Word Vectors for Sentiment Analysis Zhe Ye♠ Fang Li♠ Timothy Baldwin♥ ♠ Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China ♥ School of Computing and Information Systems, The University of Melbourne, Victoria, 3010, Australia {yezhejack, fli}@sjtu.edu.cn, tb@ldwin.net Abstract General-purpose pre-trained word embeddings have become a mainstay of natural language pro- cessing, and more recently, methods have been proposed to encode external knowledge into word embeddings to benefit specific downstream tasks. The goal of this paper is to encode sentiment knowledge into pre-trained word vectors to improve the performance of sentiment analysis. Our proposed method is based on a convolutional neural network and an external sentiment lexicon. Experiments on four popular sentiment analysis datasets show that this method improves the accuracy of sentiment analysis compared to a number of benchmark methods. 1 Introduction Sentiment analysis plays an important role in many real-world applications. The objective of sentiment classification is to classify a sentence, message or document according to sentiment, often in the form of ordinal regression (e.g. positive vs. neutral vs. negative). In recent years, deep neural networks, such as convolutional neural networks (“CNNs”), have been widely used for sentiment classification. A simple CNN trained over pre-trained word vectors has been shown to achieve highly competitive results (Kim, 2014). Learning task-specific vectors through fine-tuning may offer further gains in performance, and this is the primary focus of this paper. Separately, there has been recent work on methods for learning word embeddings based not just on textual contexts, but also external knowledge bases (Wieting et al., 2015; Yu and Dredze, 2014; Xu et al., 2014; Bian et al., 2014; Faruqui et al., 2015; Mrkˇsi´c et al., 2016; Mrkˇsi´c et al., 2017; Vuli´c et al., 2017). This has also been applied to sentiment classification (Rouvier and Favre, 2016; Yu et al., 2017), with empirical results indicating that explicitly embedding sentiment resources can improve the performance of sentiment analysis. Existing methods for encoding external knowledge into word vectors are generally trained indepen- dently of the downstream task. In order to leverage sentiment lexicons for sentiment analysis, we propose a novel method to combine a feedforward neural network (denoted “SentiNet”) with a CNN classifier to encode sentiment knowledge into word vectors during training. The method tunes word vectors through the CNN and SentiNet, based on independent information from supervised training data and sentiment lexicons. Our hypothesis is that joint training of sentiment-targeted word embeddings should improve the overall accuracy of the resulting sentiment analyzer. We conduct several experiments to verify this hypothesis, and compare our method with competitor methods that use antonymy/synonymy lexicons and paraphrase databases. The major contributions of this paper are as follows: (1) the sentiment lexicon is encoded into word vectors by a feedforward neural network instead of an objective function based on a fixed metric such as cosine similarity or Euclidean distance, …

Content truncated to 500 words