Memory Consolidation for Contextual Spoken Language Understanding with Dialogue Logistic Inference

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5448–5453 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5448 Memory Consolidation for Contextual Spoken Language Understanding with Dialogue Logistic Inference He Bai1,2, Yu Zhou1,2, Jiajun Zhang1,2 and Chengqing Zong1,2,3 1 National Laboratory of Pattern Recognition, Institute of Automation, CAS, Beijing, China 2 University of Chinese Academy of Sciences, Beijing, China 3 CAS Center for Excellence in Brain Science and Intelligence Technology {he.bai, yzhou, jjzhang, cqzong}@nlpr.ia.ac.cn Abstract Dialogue contexts are proven helpful in the spoken language understanding (SLU) system and they are typically encoded with explicit memory representations. However, most of the previous models learn the context mem- ory with only one objective to maximizing the SLU performance, leaving the context mem- ory under-exploited. In this paper, we pro- pose a new dialogue logistic inference (DLI) task to consolidate the context memory jointly with SLU in the multi-task framework. DLI is defined as sorting a shuffled dialogue ses- sion into its original logical order and shares the same memory encoder and retrieval mech- anism as the SLU model. Our experimental results show that various popular contextual SLU models can benefit from our approach, and improvements are quite impressive, espe- cially in slot filling. 1 Introduction Spoken language understanding (SLU) is a key technique in today’s conversational systems such as Apple Siri, Amazon Alexa, and Microsoft Cor- tana. A typical pipeline of SLU includes domain classification, intent detection, and slot filling(Tur and De Mori, 2011), to parse user utterances into semantic frames. Example semantic frames (Chen et al., 2018) are shown in Figure 1 for a restaurant reservation. Traditionally, domain classification and intent detection are treated as classification tasks with popular classifiers such as support vector machine and deep neural network (Haffner et al., 2003; Sarikaya et al., 2011). They can also be com- bined into one task if there are not many intents of each domain(Bai et al., 2018). Slot filling task is usually treated as a sequence labeling task. Popular approaches for slot filling include con- ditional random fields (CRF) and recurrent neu- ral network (RNN) (Raymond and Riccardi, 2007; B-time S2 S1 u1 book Which restaurant would you like to book a table for? u1 s D I restaurants reserve_restaurant a table for 10 people tonight O B-people B-date O O O O Cascal B-rest for O 6 B-people Figure 1: Example semantic frames of utterances u1 and u2 with domain (D), intent (I) and semantic slots in IOB format (S1, S2). Yao et al., 2014). Considering that pipeline ap- proaches usually suffer from error propagation, the joint model for slot filling and intent detec- tion has been proposed to improve sentence-level semantics via mutual enhancement between two tasks (Xu and Sarikaya, 2013; Hakkani-T¨ur et al., 2016; Zhang and Wang, 2016; Goo et al., 2018), which is a direction we follow. To create a more effective SLU system, the contextual information has been shown useful (Bhargava et al., 2013; Xu …

Content truncated to 500 words