Aspect Level Sentiment Classification with Deep Memory Network

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 214–224, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Aspect Level Sentiment Classification with Deep Memory Network Duyu Tang, Bing Qin∗, Ting Liu Harbin Institute of Technology, Harbin, China {dytang, qinb, tliu}@ir.hit.edu.cn Abstract We introduce a deep memory network for aspect level sentiment classification. Un- like feature-based SVM and sequential neural models such as LSTM, this approach explic- itly captures the importance of each context word when inferring the sentiment polarity of an aspect. Such importance degree and tex- t representation are calculated with multiple computational layers, each of which is a neu- ral attention model over an external memory. Experiments on laptop and restaurant datasets demonstrate that our approach performs com- parable to state-of-art feature based SVM sys- tem, and substantially better than LSTM and attention-based LSTM architectures. On both datasets we show that multiple computational layers could improve the performance. More- over, our approach is also fast. The deep mem- ory network with 9 layers is 15 times faster than LSTM with a CPU implementation. 1 Introduction Aspect level sentiment classification is a fundamen- tal task in the field of sentiment analysis (Pang and Lee, 2008; Liu, 2012; Pontiki et al., 2014). Given a sentence and an aspect occurring in the sentence, this task aims at inferring the sentiment polarity (e.g. positive, negative, neutral) of the aspect. For ex- ample, in sentence “great food but the service was dreadful!”, the sentiment polarity of aspect “food” is positive while the polarity of aspect “service” is ∗ Corresponding author. negative. Researchers typically use machine learn- ing algorithms and build sentiment classifier in a su- pervised manner. Representative approaches in liter- ature include feature based Support Vector Machine (Kiritchenko et al., 2014; Wagner et al., 2014) and neural network models (Dong et al., 2014; Lakkara- ju et al., 2014; Vo and Zhang, 2015; Nguyen and Shirai, 2015; Tang et al., 2015a). Neural models are of growing interest for their capacity to learn text representation from data without careful engineer- ing of features, and to capture semantic relations be- tween aspect and context words in a more scalable way than feature based SVM. Despite these advantages, conventional neural models like long short-term memory (LSTM) (Tang et al., 2015a) capture context information in an im- plicit way, and are incapable of explicitly exhibiting important context clues of an aspect. We believe that only some subset of context words are needed to in- fer the sentiment towards an aspect. For example, in sentence “great food but the service was dread- ful!”, “dreadful” is an important clue for the aspect “service” but “great” is not needed. Standard LST- M works in a sequential way and manipulates each context word with the same operation, so that it can- not explicitly reveal the importance of each context word. A desirable solution should be capable of ex- plicitly capturing the importance of context words and using that information to build up features for …

Content truncated to 500 words