A Lexicon-Based Supervised Attention Model for Neural Sentiment Analysis

Proceedings of the 27th International Conference on Computational Linguistics, pages 868–877 Santa Fe, New Mexico, USA, August 20-26, 2018. 868 A Lexicon-Based Supervised Attention Model for Neural Sentiment Analysis Yicheng Zou, Tao Gui, Qi Zhang, Xuanjing Huang Shanghai Key Laboratory of Intelligent Information Processing, Fudan University School of Computer Science, Fudan University 825 Zhangheng Road, Shanghai, China rowitzou@gmail.com, {tgui16,qz,xjhuang}@fudan.edu.cn Abstract Attention mechanisms have been leveraged for sentiment classification tasks because not all words have the same importance. However, most existing attention models did not take full ad- vantage of sentiment lexicons, which provide rich sentiment information and play a critical role in sentiment analysis. To achieve the above target, in this work, we propose a novel lexicon-based supervised attention model (LBSA), which allows a recurrent neural network to focus on the sen- timent content, thus generating sentiment-informative representations. Compared with general attention models, our model has better interpretability and less noise. Experimental results on three large-scale sentiment classification datasets showed that the proposed method outperforms previous methods. 1 Introduction Sentiment analysis has the goal of analyzing people’s sentiments or opinions and has been well explored (Turney, 2002; Tang et al., 2014b; Chen et al., 2016). In order to improve sentiment analysis results, large sentiment lexicons have been built (Wilson et al., 2005; Baccianella et al., 2010; Tang et al., 2014a). A sentiment lexicon is a set of words such as excellent, terrible and ordinary, each of which is assigned a fixed positive or negative score that presents its sentiment polarity and strength (Tang et al., 2014a). Such information can serve as sentiment-informative features and significantly boost the classification performance (Agarwal et al., 2013; Teng et al., 2016; Qian et al., 2016). In sentiment classification tasks, sentiment words such as great and terrible tend to play a more critical role than other words in texts (Liu, 2010). One attribute of words annotated in sentiment lexicons is their sentiment strength, which is intuitively associated with a word’s contribution to the sentiment represen- tation of a sentence. It is similar to the basic idea of the attention mechanism that not all words have the same importance. However, very few studies have focused on a method that combines an attention mechanism with such sentiment information. Yang et al. (2016) and Chen et al. (2016) proposed a hier- archical RNN model to learn attention weights based on the local context using an unsupervised method. However, their method may induce much noise and suffers from a lack of interpretability because it tends to capture some domain-specific words (Mudinas et al., 2012) instead of real sentiment information. For example, in movie reviews, the name of a movie with a good reputation tends to be regarded as positive words, and is thereby assigned higher weights, which does not work in other domains. Long et al. (2017) incorporated the reading time of human beings into the attention mechanism, but their method also has much noise because during the reading process, people tend to spend more time on intricate contents …

Content truncated to 500 words