CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6871–6883 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 6871 CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks Zixuan Ke1, Bing Liu1, Hu Xu2 and Lei Shu3∗ 1Department of Computer Science, University of Illinois at Chicago 2Facebook AI Research 3Amazon AWS AI 1{zke4,liub}@uic.edu 2huxu@fb.com 3shulindt@gmail.com Abstract This paper studies continual learning (CL) of a sequence of aspect sentiment classification (ASC) tasks in a particular CL setting called domain incremental learning (DIL). Each task is from a different domain or product. The DIL setting is particularly suited to ASC be- cause in testing the system needs not know the task/domain to which the test data belongs. To our knowledge, this setting has not been stud- ied before for ASC. This paper proposes a novel model called CLASSIC. The key novelty is a contrastive continual learning method that enables both knowledge transfer across tasks and knowledge distillation from old tasks to the new task, which eliminates the need for task ids in testing. Experimental results show the high effectiveness of CLASSIC.1 1 Introduction Continual learning (CL) learns a sequence of tasks incrementally. After learning a task, its training data is often discarded (Chen and Liu, 2018). The CL setting is useful when the data privacy is a con- cern, i.e., the data owners do not want their data used by others (Ke et al., 2020b; Qin et al., 2020; Ke et al., 2021). In such cases, if we want to lever- age the knowledge learned in the past to improve the new task learning, CL is appropriate as it shares only the learned model, but not the data. In our case, a task is a separate aspect sentiment classifi- cation (ASC) problem of a product or domain (e.g., camera or phone) (Liu, 2012). ASC is stated as follows: Given an aspect term (e.g., sound quality in a phone review) and a sentence containing the aspect (e.g., "The sound quality is poor"), ASC classifies whether the sentence expresses a positive, negative, or neutral opinion about the aspect. There are three CL settings (van de Ven and Tolias, 2019): Class Incremental Learning ∗ Work was done prior to joining Amazon. 1https://github.com/ZixuanKe/ PyContinual (CIL), Task Incremental Learning (TIL), and Do- main Incremental Learning (DIL). In CIL, the tasks contain non-overlapping classes. Only one model is built for all classes seen so far. In testing, no task information is provided. This setting is not suitable for ASC as ASC tasks have the same three classes. TIL builds one model for each task in a shared net- work. In testing, the system needs the task (e.g., phone domain) that each test instance (e.g., "The sound quality is great") belongs to and uses only the model for the task to classify the instance. Re- quiring the task information (e.g., phone domain) is a limitation. Ideally, the user should not have to provide this information for a test sentence. That is the DIL setting, …

Content truncated to 500 words