A Working Memory Model for Task-oriented Dialog Response Generation

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2687–2693 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 2687 A Working Memory Model for Task-oriented Dialog Response Generation Xiuyi Chen1,2,3, Jiaming Xu1,2∗and Bo Xu1,2,3,4 1Institute of Automation, Chinese Academy of Sciences (CASIA). Beijing, China 2Research Center for Brain-inspired Intelligence, CASIA 3University of Chinese Academy of Sciences 4Center for Excellence in Brain Science and Intelligence Technology, CAS. China {chenxiuyi2017,jiaming.xu,xubo}@ia.ac.cn Abstract Recently, to incorporate external Knowledge Base (KB) information, one form of world knowledge, several end-to-end task-oriented dialog systems have been proposed. These models, however, tend to confound the dialog history with KB tuples and simply store them into one memory. Inspired by the psycholog- ical studies on working memory, we propose a working memory model (WMM2Seq) for dialog response generation. Our WMM2Seq adopts a working memory to interact with two separated long-term memories, which are the episodic memory for memorizing dialog his- tory and the semantic memory for storing KB tuples. The working memory consists of a cen- tral executive to attend to the aforementioned memories, and a short-term storage system to store the “activated” contents from the long- term memories. Furthermore, we introduce a context-sensitive perceptual process for the to- ken representations of the dialog history, and then feed them into the episodic memory. Ex- tensive experiments on two task-oriented dia- log datasets demonstrate that our WMM2Seq significantly outperforms the state-of-the-art results in several evaluation metrics. 1 Introduction Task-oriented dialog systems, such as hotel book- ing or technical support service, help users to achieve specific goals with natural language. Com- pared with traditional pipeline solutions (Williams and Young, 2007; Young et al., 2013; Wen et al., 2017), end-to-end approaches recently gain much attention (Zhao et al., 2017; Eric and Manning, 2017a; Lei et al., 2018), because they directly map dialog history to the output responses and conse- quently reduce human effort for modular designs and hand-crafted state labels. To effectively incor- porate KB information and perform knowledge- ∗ Corresponding Author based reasoning, memory augmented models have been proposed (Bordes et al., 2017; Seo et al., 2017; Eric and Manning, 2017b; Madotto et al., 2018; Raghu et al., 2018; Reddy et al., 2019; Wu et al., 2019). Bordes et al. (2017) and Seo et al. (2017) attended to retrieval models, lacking the ability of generation, while others incorporated the memory (i.e. end-to-end memory networks, abbreviated as MemNNs, Sukhbaatar et al. (2015)) and copy mechanism (Gu et al., 2016) into a sequential gen- erative architecture. However, most models tended to confound the dialog history with KB tuples and simply stored them into one memory. A shared memory forces the memory reader to reason over the two different types of data, which makes the task harder, especially when the memory is large. To explore this problem, Reddy et al. (2019) very recently proposed to separate memories for model- ing dialog context and KB results. In this paper, we adopt working memory to interact with …

Content truncated to 500 words