Domain-Lifelong Learning for Dialogue State Tracking via Knowledge Preservation Networks

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2301–2311 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 2301 Domain-Lifelong Learning for Dialogue State Tracking via Knowledge Preservation Networks Qingbin Liu1,2, Pengfei Cao1,2, Cao Liu3, Jiansong Chen3, Xunliang Cai3, Fan Yang3, Shizhu He1,2, Kang Liu1,2, Jun Zhao1,2 1 National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China 2 School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China 3 Meituan, Beijing, China {qingbin.liu, pengfei.cao}@nlpr.ia.ac.cn, {liucao, chenjiansong, caixunliang, yangfan79}@meituan.com, {shizhu.he, kliu, jzhao}@nlpr.ia.ac.cn Abstract Dialogue state tracking (DST), which esti- mates user goals given a dialogue context, is an essential component of task-oriented dia- logue systems. Conventional DST models are usually trained offline, which requires a fixed dataset prepared in advance. This paradigm is often impractical in real-world applica- tions since online dialogue systems usually in- volve continually emerging new data and do- mains. Therefore, this paper explores Domain- Lifelong Learning for Dialogue State Tracking (DLL-DST), which aims to continually train a DST model on new data to learn incessantly emerging new domains while avoiding catas- trophically forgetting old learned domains. To this end, we propose a novel domain- lifelong learning method, called Knowledge Preservation Networks (KPN), which consists of multi-prototype enhanced retrospection and multi-strategy knowledge distillation, to solve the problems of expression diversity and com- binatorial explosion in the DLL-DST task. Ex- perimental results show that KPN effectively alleviates catastrophic forgetting and outper- forms previous state-of-the-art lifelong learn- ing methods by 4.25% and 8.27% of whole joint goal accuracy on the MultiWOZ bench- mark and the SGD benchmark, respectively. 1 Introduction Task-oriented dialogue systems aim at helping users to accomplish various tasks, such as reserving restaurants, booking flights, and checking weather (Young et al., 2013; Lei et al., 2018; Gao et al., 2020). Dialogue state tracking (DST) is an essen- tial component of task-oriented dialogue systems, which estimates user goals for downstream mod- ules (Bohus and Rudnicky, 2006; Williams et al., 2013; Henderson et al., 2014b; Mrkši´c et al., 2017; Shan et al., 2020). Given a user utterance and its Data for Restaurant DST Model (𝑓𝜃0) … Data for Flight Data for Taxi DST Model (𝑓𝜃𝑛) Figure 1: An example of domain-lifelong learning for DST. Italicized words denote domains. The DST model requires lifelong learning of new domains. dialogue history, a DST model should be able to output an accurate dialogue state. In general, the dialogue state is represented as a set of slot-value pairs, such as ((restaurant-area, north), (restaurant- price, expensive)). Previous DST models are usually trained offline, which requires a fixed dataset prepared in advance. These offline solutions are often impractical in real- world applications, as online dialogue systems usu- ally involve continually emerging new data and domains, especially when new services are intro- duced. In addition, it is infeasible to retrain DST models from scratch every time new domain data arrives due to computational costs, storage budgets, and data privacy (McMahan et al., 2017). To tackle this …

Content truncated to 500 words