Consistent Prototype Learning for Few-Shot Continual Relation Extraction

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 7409–7422 July 9-14, 2023 ©2023 Association for Computational Linguistics Consistent Prototype Learning for Few-Shot Continual Relation Extraction Xiudi Chen1,2, ∗ Hui Wu1,2, ∗ Xiaodong Shi1,2,3, † 1Department of Artificial Intelligence, School of Informatics, Xiamen University, China 2National Institute for Data Science in Health and Medicine, Xiamen University, China 3Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan (Xiamen University), Ministry of Culture and Tourism, China {chenxiudi,huistudent}@stu.xmu.edu.cn, mandel@xmu.edu.cn Abstract Few-shot continual relation extraction aims to continually train a model on incrementally few- shot data to learn new relations while avoid- ing forgetting old ones. However, current memory-based methods are prone to overfit- ting memory samples, resulting in insufficient activation of old relations and limited ability to handle the confusion of similar classes. In this paper, we design a new N-way-K-shot Continual Relation Extraction (NK-CRE) task and propose a novel few-shot continual relation extraction method with Consistent Prototype Learning (ConPL) to address the aforemen- tioned issues. Our proposed ConPL is mainly composed of three modules: 1) a prototype- based classification module that provides pri- mary relation predictions under few-shot con- tinual learning; 2) a memory-enhanced mod- ule designed to select vital samples and re- fined prototypical representations as a novel multi-information episodic memory; 3) a con- sistent learning module to reduce catastrophic forgetting by enforcing distribution consistency. To effectively mitigate catastrophic forgetting, ConPL ensures that the samples and prototypes in the episodic memory remain consistent in terms of classification and distribution. Ad- ditionally, ConPL uses prompt learning to ex- tract better representations and adopts a focal loss to alleviate the confusion of similar classes. Experimental results on two commonly-used datasets show that our model consistently out- performs other competitive baselines1. 1 Introduction Continual relation extraction (CRE) aims to con- tinually train a model on new data to learn new relations while avoiding forgetting old relations. Due to its wide applicability to real-world applica- tions, CRE has attracted increasing attention (Oba- muyide and Vlachos, 2019; Han et al., 2020; Zhao ∗Equal contribution. †Correspondence author. 1https://github.com/XiudiChen/ConPL Figure 1: The scatter plot that reflects the relation- ship between distortion and forgetting in four methods, namely EMAR (Han et al., 2020), RP-CRE (Cui et al., 2021), ERDA (Qin and Joty, 2022) and our proposed ConPL. The abscissa “distortion” indicates the degree of change in feature embedding, and the ordinate “for- getting” indicates the degree of forgetting. Each point on the plot represents a prototype of a relation class. The detailed calculations are shown in Appendix A.1. et al., 2022). Most existing studies adopt memory- based approaches as the principal architecture and achieve great success in CRE. For example, Cui et al. (2021) employ relation prototype to extract useful information of each relation. Wang et al. (2022) adopt a simple yet effective adversarial class augmentation mechanism to learn more precise and robust representations. However, obtaining large labeled data can be time-consuming and ex- …

Content truncated to 500 words