Continual Learning for Natural Language Generation in Task-oriented Dialog Systems

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3461–3474 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 3461 Continual Learning for Natural Language Generation in Task-oriented Dialog Systems Fei Mi1*, Liangwei Chen1, Mengjie Zhao2, Minlie Huang3* and Boi Faltings1 1LIA, EPFL, Lausanne, Switzerland 2CIS, LMU, Munich, Germany 3CoAI, DCST, Tsinghua University, Beijing, China {fei.mi,liangwei.chen,boi.faltings}@epfl.ch mzhao@cis.lmu.de, aihuang@tsinghua.edu.cn Abstract Natural language generation (NLG) is an es- sential component of task-oriented dialog sys- tems. Despite the recent success of neural ap- proaches for NLG, they are typically devel- oped in an offline manner for particular do- mains. To better fit real-life applications where new data come in a stream, we study NLG in a “continual learning” setting to expand its knowledge to new domains or functionalities incrementally. The major challenge towards this goal is catastrophic forgetting, meaning that a continually trained model tends to for- get the knowledge it has learned before. To this end, we propose a method called ARPER (Adaptively Regularized Prioritized Exemplar Replay) by replaying prioritized historical ex- emplars, together with an adaptive regulariza- tion technique based on Elastic Weight Consol- idation. Extensive experiments to continually learn new domains and intents are conducted on MultiWoZ-2.0 to benchmark ARPER with a wide range of techniques. Empirical results demonstrate that ARPER significantly outper- forms other methods by effectively mitigating the detrimental catastrophic forgetting issue. 1 Introduction As an essential part of task-oriented dialog systems (Wen et al., 2015b; Bordes et al., 2016), the task of Natural Language Generation (NLG) is to produce a natural language utterance containing the desired information given a semantic representation (so- called dialog act). Existing NLG models (Wen et al., 2015c; Tran and Nguyen, 2017; Tseng et al., 2018) are typically trained offline using annotated data from a single or a fixed set of domains. How- ever, a desirable dialog system in real-life applica- tions often needs to expand its knowledge to new domains and functionalities. Therefore, it is crucial to develop an NLG approach with the capability * Correspondence Author of continual learning after a dialog system is de- ployed. Specifically, an NLG model should be able to continually learn new utterance patterns without forgetting the old ones it has already learned. The major challenge of continual learning lies in catastrophic forgetting (McCloskey and Cohen, 1989; French, 1999). Namely, a neural model trained on new data tends to forget the knowledge it has acquired on previous data. We diagnose in Sec- tion 4.4 that neural NLG models suffer such detri- mental catastrophic forgetting issues when continu- ally trained on new domains. A naive solution is to retrain the NLG model using all historical data ev- ery time. However, it is not scalable due to severe computation and storage overhead. To this end, we propose storing a small set of rep- resentative utterances from previous data, namely exemplars, and replay them to the NLG model each time it needs to be trained on new data. Methods using exemplars have shown great success in differ- …

Content truncated to 500 words