Incremental Prompting: Episodic Memory Prompt for Lifelong Event Detection

Proceedings of the 29th International Conference on Computational Linguistics, pages 2157–2165 October 12–17, 2022. 2157 Incremental Prompting: Episodic Memory Prompt for Lifelong Event Detection Minqian Liu♣, Shiyu Chang♠, Lifu Huang♣ ♣Virginia Tech, ♠University of California Santa Barbara ♣{minqianliu,lifuh}@vt.edu, ♠chang87@ucsb.edu Abstract Lifelong event detection aims to incrementally update a model with new event types and data while retaining the capability on previously learned old types. One critical challenge is that the model would catastrophically forget old types when continually trained on new data. In this paper, we introduce Episodic Memory Prompts (EMP) to explicitly retain the learned task-specific knowledge. Our method adopts continuous prompt for each task and they are optimized to instruct the model prediction and learn event-specific representation. The EMPs learned in previous tasks are carried along with the model in subsequent tasks, and can serve as a memory module that keeps the old knowl- edge and transferring to new tasks. Experi- ment results demonstrate the effectiveness of our method. Furthermore, we also conduct a comprehensive analysis of the new and old event types in lifelong learning.1 1 Introduction Class-incremental event detection (Cao et al., 2020; Yu et al., 2021) is a challenging setting in lifelong learning, where the model is incrementally updated on a continual stream of data for new event types while retaining the event detection capability for all the previously learned types. The main challenge of class-incremental event detection lies in the catas- trophic forgetting problem, where the model’s per- formance on previously learned types significantly drops after it is trained on new data. Recent stud- ies (Lopez-Paz and Ranzato, 2017; Wang et al., 2019) have revealed that replaying stored samples of old classes can effectively alleviate the catas- trophic forgetting issue. However, simply fine- tuning the entire model on the limited stored sam- ples may result in overfitting, especially when the model has a huge set of parameters. How to ef- 1The source code is publicly available at https:// github.com/VT-NLP/Incremental_Prompting. fectively leverage the limited stored examples still remains an important question. Prompt learning, which is to simply tune a template-based or continuous prompt appended to the input text while keeping all the other param- eters freezed, has recently shown comparable or even better performance than fine-tuning the entire model in many NLP tasks (Brown et al., 2020; Li and Liang, 2021; Lester et al., 2021). It is espe- cially flavored by lifelong learning since it only tunes a small amount of parameters. However, it is still non-trivial to equip prompts with the capability of retaining acquired knowledge and transferring to new tasks in the class-incremental setting. In this work, we propose an incremental prompt- ing framework that introduces Epsodic Memory Prompts (EMP) to store and transfer the learned type-specific knowledge. At each training stage, we adopt a learnable prompt for each new event type added from the current task. The prompts are initialized with event type names and fine-tuned with the annotations from each task. To encour- age the prompts to always carry and reflect …

Content truncated to 500 words