How Time Matters: Learning Time-Decay Attention for Contextual Spoken Language Understanding in Dialogues

Proceedings of NAACL-HLT 2018, pages 2133–2142 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics How Time Matters: Learning Time-Decay Attention for Contextual Spoken Language Understanding in Dialogues Shang-Yu Su† Pei-Chieh Yuan† Yun-Nung Chen⋆ †Department of Electrical Engineering ⋆Department of Computer Science and Information Engineering National Taiwan University {r05921117,b03901134}@ntu.edu.tw y.v.chen@ieee.org Abstract Spoken language understanding (SLU) is an essential component in conversational sys- tems. Most SLU components treat each ut- terance independently, and then the following components aggregate the multi-turn informa- tion in the separate phases. In order to avoid error propagation and effectively utilize con- texts, prior work leveraged history for con- textual SLU. However, most previous models only paid attention to the related content in his- tory utterances, ignoring their temporal infor- mation. In the dialogues, it is intuitive that the most recent utterances are more important than the least recent ones, in other words, time- aware attention should be in a decaying man- ner. Therefore, this paper designs and investi- gates various types of time-decay attention on the sentence-level and speaker-level, and fur- ther proposes a flexible universal time-decay attention mechanism. The experiments on the benchmark Dialogue State Tracking Chal- lenge (DSTC4) dataset show that the proposed time-decay attention mechanisms significantly improve the state-of-the-art model for contex- tual understanding performance1. 1 Introduction Spoken dialogue systems that can help users to solve complex tasks such as booking a movie ticket have become an emerging research topic in artificial intelligence and natural language pro- cessing areas. With a well-designed dialogue sys- tem as an intelligent personal assistant, people can accomplish certain tasks more easily via natural language interactions. Today, there are several virtual intelligent assistants, such as Apple’s Siri, Google’s Home, Microsoft’s Cortana, and Ama- zon’s Echo. Recent advance of deep learning has 1The source code is at: https://github.com/ MiuLab/Time-Decay-SLU. inspired many applications of neural models to di- alogue systems (Wen et al., 2017; Bordes et al., 2017; Dhingra et al., 2017; Li et al., 2017). A key component of a dialogue system is a spoken language understanding (SLU) module— it parses user utterances into semantic frames that capture the core meaning (Tur and De Mori, 2011). A typical pipeline of SLU is to first de- cide the domain given the input utterance, and based on the domain, to predict the intent and to fill associated slots corresponding to a domain- specific semantic template, where each utterance is treated independently (Hakkani-T¨ur et al., 2016; Chen et al., 2016b,a; Wang et al., 2016). To over- come the error propagation and further improve understanding performance, the contextual infor- mation has been shown useful (Bhargava et al., 2013; Xu and Sarikaya, 2014; Chen et al., 2015; Sun et al., 2016). Prior work incorporated the di- alogue history into the recurrent neural networks (RNN) for improving domain classification, intent prediction, and slot filling (Xu and Sarikaya, 2014; Shi et al., 2015; Weston et al., 2015; Chen et al., 2016c). Recently, Chi et al. (2017) and Zhang et al. (2018) demonstrated …

Content truncated to 500 words