An LSTM Approach to Short Text Sentiment Classification with Word Embeddings

The 2018 Conference on Computational Linguistics and Speech Processing ROCLING 2018, pp. 214-223 ©The Association for Computational Linguistics and Chinese Language Processing An LSTM Approach to Short Text Sentiment Classification with Word Embeddings Jenq-Haur Wang Department of Computer Science and Information Engineering National Taipei University of Technology, Taipei, Taiwan jhwang@csie.ntut.edu.tw Ting-Wei Liu PChome eBay Co., Ltd., Taipei, Taiwan s5548765@gmail.com Xiong Luo, Long Wang School of Computer and Communication Engineering University of Science and Technology Beijing, Beijing, China xluo@ustb.edu.cn, lwang@ustb.edu.cn Abstract Sentiment classification techniques have been widely used for analyzing user opinions. In conventional supervised learning methods, hand-crafted features are needed, which requires a thorough understanding of the domain. Since social media posts are usually very short, there’s a lack of features for effective classification. Thus, word embedding models can be used to learn different word usages in various contexts. To detect the sentiment polarity from short texts, we need to explore deeper semantics of words using deep learning methods. In this paper, we investigate the effects of word embedding and long short-term memory (LSTM) for sentiment classification in social media. First, words in posts are converted into vectors using word embedding models. Then, the word sequence in sentences are input to LSTM to learn the long distance contextual dependency among words. The experimental results showed that deep learning methods can effectively learn the word usage in context of social media given enough training data. The quantity and quality of training data greatly affects the performance. Further investigation is needed to verify the performance in different social media sources. Keywords: Sentiment Classification, Deep Learning, Long Short-Term Memory, Word2Vec Model. 214 1. Introduction Sentiment classification has been used in analyzing user-generated contents for understanding users’ intent and opinions in social media. Conventional supervised learning methods have been extensively investigated such as bag-of-words model using TF-IDF, and probabilistic model using Naïve Bayes, which usually need hand-crafted features. For social media content which are very short and diverse in topic, it’s difficult to obtain useful features for classification. Thus, a more effective method for short text sentiment classification is needed. Deep learning methods have gradually shown good performance in many applications, such as speech recognition, pattern recognition, and data classification. These methods try to learn data representation using a deeper hierarchy of structures in neural networks. Complicated concepts are possible to learn based on simpler ones. Among deep feedforward networks, Convolutional Neural Networks (CNNs) have been shown to learn local features from words or phrases [1], while Recurrent Neural Networks (RNNs) are able to learn temporal dependencies in sequential data [2]. Given the very short texts in social media, there’s a lack of features. To obtain more useful features, we further utilize the idea of distributed representation of words where each input is represented by many features and each feature is involved in many possible inputs. Specifically, we use the Word2Vec word embedding model [3] for distributed representation of social posts. In this paper, we want to investigate the effectiveness of long short-term memory …

Content truncated to 500 words