Continual Few-shot Relation Learning via Embedding Space Regularization and Data Augmentation

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 2776 - 2789 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Continual Few-shot Relation Learning via Embedding Space Regularization and Data Augmentation Chengwei Qin♣ and Shafiq Joty♣♠ ♣ Nanyang Technological University ♠ Salesforce Research {chengwei003@e.ntu, srjoty@ntu}.edu.sg Abstract Existing continual relation learning (CRL) methods rely on plenty of labeled training data for learning a new task, which can be hard to acquire in real scenario as getting large and representative labeled data is often expensive and time-consuming. It is therefore necessary for the model to learn novel relational patterns with very few labeled data while avoiding catas- trophic forgetting of previous task knowledge. In this paper, we formulate this challenging yet practical problem as continual few-shot re- lation learning (CFRL). Based on the finding that learning for new emerging few-shot tasks often results in feature distributions that are incompatible with previous tasks’ learned dis- tributions, we propose a novel method based on embedding space regularization and data aug- mentation. Our method generalizes to new few- shot tasks and avoids catastrophic forgetting of previous tasks by enforcing extra constraints on the relational embeddings and by adding extra relevant data in a self-supervised manner. With extensive experiments we demonstrate that our method can significantly outperform previous state-of-the-art methods in CFRL task settings.1 1 Introduction Relation Extraction (RE) aims to detect the re- lationship between two entities in a sentence, for example, predicting the relation birthdate in the sentence “Kamala Harris was born in Oakland, California, on October 20, 1964.” for the two enti- ties Kamala Harris and October 20, 1964. It serves as a fundamental step for downstream tasks such as search and question answering (Dong et al., 2015; Yu et al., 2017). Traditionally, RE methods were built by considering a fixed static set of relations (Miwa and Bansal, 2016; Han et al., 2018a). How- ever, similar to entity recognition, RE is also an open-vocabulary problem (Sennrich et al., 2016), 1Code and models are available at https://github.com/qcwthu/Continual_Fewshot_Relation_Learning Figure 1: Difference between Continual Relation Learn- ing (CRL) and Continual Few-shot Relation Learning (CFRL). Except for the first task which has enough train- ing data, the subsequent new tasks are all few-shot in CFRL. In contrast, CRL assumes enough training data for every task. where the relation set keeps growing as new rela- tion types emerge with new data. A potential solution is to formalize RE as Contin- ual Relation Learning or CRL (Wang et al., 2019). In CRL, the model learns relational knowledge through a sequence of tasks, where the relation set changes dynamically from the current task to the next. The model is expected to perform well on both the novel and previous tasks, which is chal- lenging due to the existence of Catastrophic Forget- ting phenomenon (McCloskey and Cohen, 1989; French, 1999) in continual learning. In this phe- nomenon, the model forgets previous relational knowledge after learning new relational patterns. Existing methods to address catastrophic …

Content truncated to 500 words