Progressive Self-Supervised Attention Learning for Aspect-Level Sentiment Analysis

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 557–566 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 557 Progressive Self-Supervised Attention Learning for Aspect-Level Sentiment Analysis Jialong Tang1,2,3∗, Ziyao Lu1∗, Jinsong Su1†, Yubin Ge4, Linfeng Song5, Le Sun2, Jiebo Luo5 1Xiamen University, Xiamen, China 2Institute of Software, Chinese Academy of Sciences, Beijing, China 3University of Chinese Academy of Sciences, Beijing, China 4University of Illinois at Urbana-Champaign, Urbana, IL 61801, USA 5Department of Computer Science, University of Rochester, Rochester NY 14627, USA jialong2019@iscas.ac.cn, ziyaolu2018@stu.xmu.edu.cn jssu@xmu.edu.cn Abstract In aspect-level sentiment classification (ASC), it is prevalent to equip dominant neural mod- els with attention mechanisms, for the sake of acquiring the importance of each context word on the given aspect. However, such a mecha- nism tends to excessively focus on a few fre- quent words with sentiment polarities, while ignoring infrequent ones. In this paper, we propose a progressive self-supervised atten- tion learning approach for neural ASC mod- els, which automatically mines useful atten- tion supervision information from a training corpus to refine attention mechanisms. Specif- ically, we iteratively conduct sentiment predic- tions on all training instances. Particularly, at each iteration, the context word with the maxi- mum attention weight is extracted as the one with active/misleading influence on the cor- rect/incorrect prediction of every instance, and then the word itself is masked for subsequent iterations. Finally, we augment the conven- tional training objective with a regularization term, which enables ASC models to continue equally focusing on the extracted active con- text words while decreasing weights of those misleading ones. Experimental results on mul- tiple datasets show that our proposed approach yields better attention mechanisms, leading to substantial improvements over the two state- of-the-art neural ASC models. Source code and trained models are available.1 1 Introduction Aspect-level sentiment classification (ASC), as an indispensable task in sentiment analysis, aims at inferring the sentiment polarity of an input sen- tence in a certain aspect. In this regard, pre- ∗Equal contribution †Corresponding author 1https://github.com/DeepLearnXMU/PSSAttention vious representative models are mostly discrim- inative classifiers based on manual feature engi- neering, such as Support Vector Machine (Kir- itchenko et al., 2014; Wagner et al., 2014). Re- cently, with the rapid development of deep learn- ing, dominant ASC models have evolved into neural network (NN) based models (Tang et al., 2016b; Wang et al., 2016; Tang et al., 2016a; Ma et al., 2017; Chen et al., 2017; Li et al., 2018; Wang et al., 2018), which are able to automati- cally learn the aspect-related semantic representa- tion of an input sentence and thus exhibit better performance. Usually, these NN-based models are equipped with attention mechanisms to learn the importance of each context word towards a given aspect. It can not be denied that attention mecha- nisms play vital roles in neural ASC models. However, the existing attention mechanism in ASC suffers from a major drawback. Specifically, it is prone to overly focus on a few frequent words with sentiment polarities …

Content truncated to 500 words