Overcoming Catastrophic Forgetting During Domain Adaptation of Seq2seq Language Generation

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5441 - 5454 July 10-15, 2022 ©2022 Association for Computational Linguistics Overcoming Catastrophic Forgetting During Domain Adaptation of Seq2seq Language Generation Dingcheng Li, Zheng Chen, Eunah Cho, Jie Hao, Xiaohu Liu, Xing Fan, Chenlei Guo, Yang Liu Amazon Alexa AI Seattle, Washington, USA zgchen,eunahch,jieha,derekliu fanxing,guochenl,yangliud@amazon.com Abstract Seq2seq language generation models that are trained offline with multiple domains in a se- quential fashion often suffer from catastrophic forgetting. Lifelong learning has been pro- posed to handle this problem. However, exist- ing work such as experience replay or elas- tic weighted consolidation requires incremen- tal memory space. In this work, we propose an innovative framework, RMR_DSE that lever- ages a recall optimization mechanism to selec- tively memorize important parameters of pre- vious tasks via regularization, and uses a do- main drift estimation algorithm to compensate for the drift between different domains in the embedding space. These designs enable the model to be trained on the current task while keeping the memory of previous tasks, and avoid much additional data storage. Further- more, RMR_DSE can be combined with ex- isting lifelong learning approaches. Our exper- iments on two seq2seq language generation tasks, paraphrase and dialog response genera- tion, show that RMR_DSE outperforms state- of-the-art models by a considerable margin and greatly reduces forgetting. 1 Introduction Seq2seq language generation is the essential frame- work for many tasks such as machine translation, summarization, paraphrase, question answering, di- alog response generation. In these applications, models are typically trained offline using anno- tated data from a fixed set of domains. However, in real-world applications, it is desirable for the system to expand its knowledge to new domains and functionalities, that is, it has the capability of human-like lifelong learning (LLL) (Ring et al., 1994; Chaudhry et al., 2019) of acquiring new utter- ance patterns without forgetting what it has already learned. Neural networks struggle to learn continu- ously and experience catastrophic forgetting (CF) when optimized on a sequence of learning prob- lems (McCloskey and Cohen, 1989; French, 1999). Some past work in LLL has demonstrated that dis- criminative models can be incrementally learnt for a sequence of tasks (Kirkpatrick et al., 2017; Chen et al., 2020; Wang et al., 2019). In contrast, under generative settings there has been limited research. Recent work in this area includes (Mi et al., 2020; Madotto et al., 2020; He et al., 2021; Shin et al., 2017). Existing work in LLL adopts the replay based methods (Pellegrini et al., 2019), such as Latent Replay, or regularization based methods (Huszár, 2018; Li and Hoiem, 2018), such as Elastic Weight Consolidation (EWC) (Kirkpatrick et al., 2017), Synaptic Intelligence (Zenke et al., 2017). Al- though they can reduce CF, they have some limi- tations. The replay-based methods require storing samples from previous tasks, and regularization methods often view all the model parameters as equally important and regularize them to the same extent. In addition, those …

Content truncated to 500 words