Generative Replay Inspired by Hippocampal Memory Indexing for Continual Language Learning

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 930–942 May 2-6, 2023 ©2023 Association for Computational Linguistics Generative Replay Inspired by Hippocampal Memory Indexing for Continual Language Learning Aru Maekawa1, Hidetaka Kamigaito1,2, Kotaro Funakoshi1, Manabu Okumura1 1Tokyo Institute of Technology, 2Nara Institute of Science and Technology {maekawa, kamigaito, funakoshi, oku}@lr.pi.titech.ac.jp kamigaito.h@is.naist.jp Abstract Continual learning aims to accumulate knowl- edge to solve new tasks without catastrophic forgetting for previously learned tasks. Re- search on continual learning has led to the de- velopment of generative replay, which prevents catastrophic forgetting by generating pseudo- samples for previous tasks and learning them together with new tasks. Inspired by the bi- ological brain, we propose the hippocampal memory indexing to enhance the generative re- play by controlling sample generation using compressed features of previous training sam- ples. It enables the generation of a specific training sample from previous tasks, thus im- proving the balance and quality of generated replay samples. Experimental results indicate that our method effectively controls the sam- ple generation and consistently outperforms the performance of current generative replay meth- ods.1 1 Introduction Humans and intelligent animals continually acquire new knowledge and skills throughout their life- time. This ability, called continual learning (CL) or lifelong learning, is a fundamental requirement for human-like general intelligence (Parisi et al., 2019). CL is also crucial for practical applications, as new data and tasks to train models appear every day in the real world. It is especially important for natural language processing (NLP) systems, in which vocabulary and language usage change over time. However, most neural network based mod- els are trained with a static dataset. When learning different tasks sequentially, performance on the pre- viously learned tasks tends to significantly degrade, referred to as catastrophic forgetting (McCloskey and Cohen, 1989). Learning new tasks without catastrophic forgetting has been a long-standing challenge in machine learning and neural networks. 1The source code is available at https://github. com/arumaekawa/GR-HMI. Replay is an approach to alleviate catastrophic forgetting by retraining with previous tasks’ data when training a new task. Although replay-based methods are effective in most CL scenarios, it is necessary to retain training data for all previous tasks, which may cause problems with storage re- quirements and data privacy. Therefore, generative replay was developed (Shin et al., 2017), which uses pseudo-samples generated from generation models instead of real samples. In the NLP do- main, LAMOL (Sun et al., 2020a) was proposed as a generative replay framework, where a single lan- guage model simultaneously learns to solve tasks and to generate pseudo-samples (Fig. 1a). Although the generative replay does not require any previous task’s data, it typically underperforms actual sample replay because of the balance and quality problems in sample generation. Regarding the balance problem, generation models tend to generate a large number of samples for recently learned tasks, that results in forgetting older tasks. This is due to the difficulty in controlling sample generation and catastrophic forgetting occurring in generation models. Regarding …

Content truncated to 500 words