Linguistically Regularized LSTM for Sentiment Classification

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1679–1689 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1154 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1679–1689 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1154 Linguistically Regularized LSTM for Sentiment Classification Qiao Qian1, Minlie Huang1∗, Jinhao Lei2, Xiaoyan Zhu1 1State Key Laboratory of Intelligent Technology and Systems Tsinghua National Laboratory for Information Science and Technology Dept. of Computer Science and Technology, Tsinghua University, Beijing 100084, PR China 2Dept. of Thermal Engineering, Tsinghua University, Beijing 100084, PR China qianqiaodecember29@126.com, aihuang@tsinghua.edu.cn leijh14@gmail.com , zxy-dcs@tsinghua.edu.cn Abstract This paper deals with sentence-level sen- timent classification. Though a variety of neural network models have been pro- posed recently, however, previous models either depend on expensive phrase-level annotation, most of which has remark- ably degraded performance when trained with only sentence-level annotation; or do not fully employ linguistic resources (e.g., sentiment lexicons, negation words, inten- sity words). In this paper, we propose sim- ple models trained with sentence-level an- notation, but also attempt to model the lin- guistic role of sentiment lexicons, nega- tion words, and intensity words. Results show that our models are able to cap- ture the linguistic role of sentiment words, negation words, and intensity words in sentiment expression. 1 Introduction Sentiment classification aims to classify text to sentiment classes such as positive or negative, or more fine-grained classes such as very positive, positive, neutral, etc. There has been a variety of approaches for this purpose such as lexicon-based classification (Turney, 2002; Taboada et al., 2011), and early machine learning based methods (Pang et al., 2002; Pang and Lee, 2005), and recently neural network models such as convolutional neu- ral network (CNN) (Kim, 2014; Kalchbrenner et al., 2014; Lei et al., 2015), recursive autoen- coders (Socher et al., 2011, 2013), Long Short- Term Memory (LSTM) (Mikolov, 2012; Chung et al., 2014; Tai et al., 2015; Zhu et al., 2015), and many more. ∗Corresponding Author: Minlie Huang In spite of the great success of these neural mod- els, there are some defects in previous studies. First, tree-structured models such as recursive au- toencoders and Tree-LSTM (Tai et al., 2015; Zhu et al., 2015), depend on parsing tree structures and expensive phrase-level annotation, whose per- formance drops substantially when only trained with sentence-level annotation. Second, linguis- tic knowledge such as sentiment lexicon, negation words or negators (e.g., not, never), and intensity words or intensifiers (e.g., very, absolutely), has not been fully employed in neural models. The goal of this research is to developing sim- ple sequence models but also attempts to fully em- ploying linguistic resources to benefit sentiment classification. Firstly, we attempts to develop sim- ple models that do not depend on parsing trees and do not require phrase-level annotation which is too expensive in real-world applications. Secondly, in order to obtain competitive performance, sim- ple models can benefit from linguistic resources. Three …

Content truncated to 500 words