Learning Robust Representations for Continual Relation Extraction via Adversarial Class Augmentation

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6264 - 6278 December 7-11, 2022 ©2022 Association for Computational Linguistics Learning Robust Representations for Continual Relation Extraction via Adversarial Class Augmentation Peiyi Wang1∗ Yifan Song1∗ Tianyu Liu2 Binghuai Lin2 Yunbo Cao2 Sujian Li1 Zhifang Sui1 1 MOE Key Laboratory of Computational Linguistics, Peking University, China 2 Tencent Cloud Xiaowei wangpeiyi9979@gmail.com; {yfsong, lisujian, szf}@pku.edu.cn {rogertyliu, binghuailin, yunbocao}@tencent.com; Abstract Continual relation extraction (CRE) aims to continually learn new relations from a class- incremental data stream. CRE model usually suffers from catastrophic forgetting problem, i.e., the performance of old relations seriously degrades when the model learns new relations. Most previous work attributes catastrophic for- getting to the corruption of the learned repre- sentations as new relations come, with an im- plicit assumption that the CRE models have adequately learned the old relations. In this paper, through empirical studies we argue that this assumption may not hold, and an impor- tant reason for catastrophic forgetting is that the learned representations do not have good robustness against the appearance of analogous relations in the subsequent learning process. To address this issue, we encourage the model to learn more precise and robust representa- tions through a simple yet effective adversarial class augmentation mechanism (ACA), which is easy to implement and model-agnostic. Ex- perimental results show that ACA can consis- tently improve the performance of state-of-the- art CRE models on two popular benchmarks. Our code is available at https://github. com/Wangpeiyi9979/ACA. 1 Introduction Relation extraction (RE) aims to detect the relation of two given entities in a sentence. Traditional RE models are trained on a fixed dataset with a pre- defined relation set, which cannot handle the real- life situation where new relations are constantly emerging. To this end, continual relation extraction (CRE) (Wang et al., 2019; Han et al., 2020; Cui et al., 2021; Zhao et al., 2022; Wang et al., 2022) is introduced. As shown in Figure 1, CRE is formu- lated as a class-incremental problem, which trains the model on a sequence of tasks. In each task, *Equal contribution. R1,R2 task1 R1,R2 R1,R2 R3,R4 R1,R2 R3,R4 R5,R6 R3,R4 R5,R6 R1: child R2: architecture R3: location R4: employee R5: father R6: characters task2 task3 train test easy dataset continually learn new relations hard dataset Figure 1: A demonstration for continual relation extrac- tion with three tasks where each task involves two new relations. The learned representations from the easy training task can not handle the hard testing data, which contains analogous relations inherently hard to distin- guish, e.g., “child” and “father”. the model needs to learn some new relations and is evaluated on all seen relations. Like other con- tinual learning systems, CRE models also suffer from catastrophic forgetting, i.e., the performance of previously learned relations seriously degrades when learning new relations. The mainstream research in CRE (Han et al., 2020; Cui et al., 2021; Zhao et al., 2022; Wang et al., 2022) mainly attributes catastrophic forget- ting to the corruption of …

Content truncated to 500 words