CAN: Constrained Attention Networks for Multi-Aspect Sentiment Analysis

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 4601–4610, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 4601 CAN: Constrained Attention Networks for Multi-Aspect Sentiment Analysis Mengting Hu1∗ Shiwan Zhao2† Li Zhang2 Keke Cai2 Zhong Su2 Renhong Cheng1 Xiaowei Shen2 1 Nankai University 2 IBM Research - China mthu@mail.nankai.edu.cn, {zhaosw, lizhang, caikeke, suzhong}@cn.ibm.com chengrh@nankai.edu.cn, xwshen@cn.ibm.com Abstract Aspect level sentiment classification is a fine- grained sentiment analysis task. To detect the sentiment towards a particular aspect in a sentence, previous studies have developed various attention-based methods for generat- ing aspect-specific sentence representations. However, the attention may inherently intro- duce noise and downgrade the performance. In this paper, we propose constrained atten- tion networks (CAN), a simple yet effective solution, to regularize the attention for multi- aspect sentiment analysis, which alleviates the drawback of the attention mechanism. Specif- ically, we introduce orthogonal regularization on multiple aspects and sparse regularization on each single aspect. Experimental results on two public datasets demonstrate the effective- ness of our approach. We further extend our approach to multi-task settings and outperform the state-of-the-art methods. 1 Introduction Sentiment analysis (Nasukawa and Yi, 2003; Liu, 2012), an important task in natural language understanding, receives much attention in re- cent years. Aspect level sentiment classification (ALSC) is a fine-grained sentiment analysis task, which aims at detecting the sentiment towards a particular aspect in a sentence. ALSC is es- pecially critical for multi-aspect sentences which contain multiple aspects. A multi-aspect sen- tence can be categorized as overlapping or non- overlapping. A sentence is annotated as non- overlapping if any two of its aspects have no over- lap. Our study found that around 85% of the multi- aspect sentences are non-overlapping in the two public datasets. Figure 1 shows a simple exam- ple. The non-overlapping sentence contains two ∗This work was done when Mengting Hu was a research intern at IBM Research - China. †Corresponding author. Aspect category: service, polarity: neutral Aspect category: food, polarity: positive 0.17 0.11 0.07 0.19 sometimes i get good food and ok service . Figure 1: Example of a non-overlapping sentence. The attention weights of the aspect food are from the model ATAE-LSTM (Wang et al., 2016). aspects. The aspect food is on the left side of the aspect service. Their distributions on words are orthogonal to each other. Another observation is that only a few words relate to the opinion expres- sion in each aspect. As shown in Figure 1, only the word “good” is relevant to the aspect food and “ok” to service. The distribution of the opinion expression of each aspect is sparse. To detect the sentiment towards a particular as- pect, previous studies (Wang et al., 2016; Ma et al., 2017; Cheng et al., 2017; Ma et al., 2018; Huang et al., 2018) have developed various attention- based methods for generating aspect-specific sen- tence representations. In these works, the attention may inherently introduce noise and …

Content truncated to 500 words