Human-Like Decision Making: Document-level Aspect Sentiment Classification via Hierarchical Reinforcement Learning

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 5581–5590, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 5581 Human-Like Decision Making: Document-level Aspect Sentiment Classification via Hierarchical Reinforcement Learning Jingjing Wang1, Changlong Sun2, Shoushan Li1,∗, Jiancheng Wang1, Luo Si2, Min Zhang1, Xiaozhong Liu2, Guodong Zhou1 1School of Computer Science and Technology, Soochow University, China 2Alibaba Group, China {djingwang, lishoushan, minzhang, gdzhou}@suda.edu.cn, jiancheng.wang@qq.com, {changlong.scl, xiaozhong.lxz, luo.si}@alibaba-inc.com Abstract Recently, neural networks have shown promis- ing results on Document-level Aspect Senti- ment Classification (DASC). However, these approaches often offer little transparency w.r.t. their inner working mechanisms and lack in- terpretability. In this paper, to simulating the steps of analyzing aspect sentiment in a docu- ment by human beings, we propose a new Hi- erarchical Reinforcement Learning (HRL) ap- proach to DASC. This approach incorporates clause selection and word selection strategies to tackle the data noise problem in the task of DASC. First, a high-level policy is pro- posed to select aspect-relevant clauses and dis- card noisy clauses. Then, a low-level policy is proposed to select sentiment-relevant words and discard noisy words inside the selected clauses. Finally, a sentiment rating predictor is designed to provide reward signals to guide both clause and word selection. Experimental results demonstrate the impressive effective- ness of the proposed approach to DASC over the state-of-the-art baselines. 1 Introduction Document-level Aspect Sentiment Classification (DASC) is a fine-grained sentiment classification task in the field of sentiment analysis (Pang and Lee, 2007; Li et al., 2010). This task aims to predict the sentiment rating for each given aspect mentioned in a document-level review. For in- stance, Figure 1 shows a review document with four given aspects of a hotel (i.e., location, room, value, service). The goal of DASC is to predict the rating score towards each aspect by analyz- ing the whole document. In the last decade, this task has been drawing more and more interests of researchers in the Natural Language Processing community (Titov and McDonald, 2008; Yin et al., 2017; Li et al., 2018). In previous studies, neu- ∗Corresponding author Review Document [[[This hotel is close to railway station]]]Clause1 [[[and very convenient to eat around]]]Clause2 [[[but room of Hilton is a little uncomfortable.]Clause3 [[[I’m often nitpicking for room decoration.]]]Clause4 [[[Besides, the price is very expensive ]]]Clause5 [[[although the staff service is professional.]]]Clause6 Rating of Each Aspect - location: # # # # # (5) - room: # # # $ $ (3) - value: # $ $ $ $ (1) - service: # # # # $ (4) Figure 1: An example of a review document, where clauses and words with different colors refer to different aspects. ral models have shown to be effective for perfor- mance improvement on DASC. Despite the advan- tages, these complex neural network approaches often offer little transparency w.r.t. their inner working mechanisms and suffer from the lack of interpretability. However, clearly understanding where and how such …

Content truncated to 500 words