Using the Past Knowledge to Improve Sentiment Classification

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1124–1133 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 1124 Using the Past Knowledge to Improve Sentiment Classification Qi Qin1,2 , Wenpeng Hu3, Bing Liu2,∗ 1 Center for Data Science, AAIS, Peking University 2 Wangxuan Institute of Computer Technology, Peking University 3 Department of Information Science, Peking University {qinqi, wenpeng.hu}@pku.edu.cn, dcsliub@gmail.com Abstract This paper studies sentiment classification in the lifelong learning setting that incremen- tally learns a sequence of sentiment classifi- cation tasks. It proposes a new lifelong learn- ing model (called L2PG) that can retain and selectively transfer the knowledge learned in the past to help learn the new task. A key in- novation of this proposed model is a novel parameter-gate (p-gate) mechanism that reg- ulates the flow or transfer of the previously learned knowledge to the new task. Specifi- cally, it can selectively use the network param- eters (which represent the retained knowledge gained from the previous tasks) to assist the learning of the new task t. Knowledge distilla- tion is also employed in the process to preserve the past knowledge by approximating the net- work output at the state when task t − 1 was learned. Experimental results show that L2PG outperforms strong baselines, including even multiple task learning. 1 Introduction A typical sentiment analysis (SA) or social media company that provides sentiment analysis services has to work for a large number of clients (Liu, 2012). Each client normally wants to study peo- ple’s opinions about a particular category of prod- ucts or services, which we also call a domain. If we regard each such study/project as a task, we can model a SA company’s working on a large number of studies/projects for clients as perform- ing a sequence of SA tasks. A natural question that one would ask is whether after analyzing opin- ions about a number of products or services (tasks), the SA system of the company can do better on a new task by retaining the knowledge learned from the past/previous tasks and selectively transfer the ∗Corresponding Author. This work was done when this author was on leave at Peking University. His current affilia- tion is University of Illinois at Chicago: liub@uic.edu. prior knowledge to the new task to help it learn bet- ter. The answer should be yes because words and phrases used to express opinions or sentiments in different domains are similar and thus can mostly be shared or transferred across domains, although different domains do have domain specific senti- ment expressions. This is a lifelong learning set- ting (Thrun, 1998; Silver et al., 2013; Chen and Liu, 2016). This paper focuses on lifelong senti- ment classification (Chen et al., 2015). Problem Definition: We consider incremen- tally learning a sequence of supervised sentiment classification (SC) tasks, 1, ..., t, .... Each task t has a training dataset Dt train = {xt i, yt i}nt i=1, where xt i is an input instance and yt i is its label, and …

Content truncated to 500 words