A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understanding

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 2078–2087, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 2078 A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understanding Libo Qin, Wanxiang Che∗, Yangming Li, Haoyang Wen, Ting Liu Research Center for Social Computing and Information Retrieval Harbin Institute of Technology, China {lbqin,car,yangmingli,hywen,tliu}@ir.hit.edu.cn Abstract Intent detection and slot filling are two main tasks for building a spoken language under- standing (SLU) system. The two tasks are closely tied and the slots often highly de- pend on the intent. In this paper, we pro- pose a novel framework for SLU to better in- corporate the intent information, which fur- ther guides the slot filling. In our frame- work, we adopt a joint model with Stack- Propagation which can directly use the intent information as input for slot filling, thus to capture the intent semantic knowledge. In addition, to further alleviate the error propa- gation, we perform the token-level intent de- tection for the Stack-Propagation framework. Experiments on two publicly datasets show that our model achieves the state-of-the-art performance and outperforms other previous methods by a large margin. Finally, we use the Bidirectional Encoder Representation from Transformer (BERT) model in our framework, which further boost our performance in SLU task. 1 Introduction Spoken language understanding (SLU) is a criti- cal component in task-oriented dialogue systems. It usually consists of intent detection to identify users’ intents and slot filling task to extract se- mantic constituents from the natural language ut- terances (Tur and De Mori, 2011). As shown in Table 1, given a movie-related utterance “watch action movie”, there are different slot labels for each token and an intent for the whole utterance. Usually, intent detection and slot filling are implemented separately. But intuitively, these two tasks are not independent and the slots of- ten highly depend on the intent (Goo et al., 2018). For example, if the intent of a utterance is ∗* Email corresponding. Sentence watch action movie Gold Slots O B-movie name I-movie name Gold Intent WatchMovie Table 1: An example with intent and slot annotation (BIO format), which indicates the slot of movie name from an utterance with an intent WatchMovie. WatchMovie, it is more likely to contain the slot movie name rather than the slot music name. Hence, it is promising to incorporate the intent in- formation to guide the slot filling. Considering this strong correlation between the two tasks, some joint models are proposed based on the multi-task learning framework (Zhang and Wang, 2016; Hakkani-T¨ur et al., 2016; Liu and Lane, 2016) and all these models outperform the pipeline models via mutual enhancement between two tasks. However, their work just modeled the relationship between intent and slots by shar- ing parameters. Recently, some work begins to model the intent information for slot filling ex- plicitly in joint model. Goo et al. (2018) and Li et al. (2018) proposed the …

Content truncated to 500 words