SlotRefine: A Fast Non-Autoregressive Model for Joint Intent Detection and Slot Filling

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 1932–1937, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 1932 SlotRefine: A Fast Non-Autoregressive Model for Joint Intent Detection and Slot Filling Di Wu§ Liang Ding† Fan Lu♭ Jian Xie‡ §Peking University ♭University of Washington ‡Wuhan University inbath@163.com lufan0929@gmail.com xiejian1990@gmail.com †UBTECH Sydney AI Centre, School of Computer Science Faculty of Engineering, The University of Sydney ldin3097@uni.sydney.edu.au Abstract Slot filling and intent detection are two main tasks in spoken language understanding (SLU) system. In this paper, we propose a novel non-autoregressive model named SlotRefine for joint intent detection and slot filling. Be- sides, we design a novel two-pass iteration mechanism to handle the uncoordinated slots problem caused by conditional independence of non-autoregressive model. Experiments demonstrate that our model significantly out- performs previous models in slot filling task, while considerably speeding up the decoding (up to ×10.77). In-depth analyses show that 1) pretraining schemes could further enhance our model; 2) two-pass mechanism indeed remedy the uncoordinated slots. 1 Introduction Slot filling (SF) and intent detection (ID) play im- portant roles in spoken language understanding, especially for task-oriented dialogue system. For example, for an utterance like “Buy an air ticket from Beijing to Seattle”, intent detection works on sentence-level to indicate the task is about purchas- ing an air ticket, while the slot filling focus on words-level to figure out the departure and destina- tion of that ticket are “Beijing” and “Seattle”. In early studies, ID and SF were often modeled separately, where ID was modeled as a classifica- tion task, while SF was regarded as a sequence labeling task. Due to the correlation between these two tasks, training them jointly could enhance each other. Zhang and Wang (2016) propose a joint model using bidirectional gated recurrent unit to learn the representation at each time step. Mean- while, a max-pooling layer is employed to capture the global features of a sentence for intent classi- fication. Liu and Lane (2016) cast the slot filling task as a tag generation problem and introduce a recurrent neural network based encoder-decoder framework with attention mechanism to model it, meanwhile using the encoded vector to predict in- tent. Goo et al. (2018) and Haihong et al. (2019) dig into the correlation between ID and SF deeper and modeled the relationship between them explic- itly. Qin et al. (2019) propagate the token-level intent results to the SF task, achieving significant performance improvement. Briefly summarized, most of the previous works heavily rely on autoregressive approaches, e.g., RNN based model or seq2seq architecture, to cap- ture the grammar structure in an utterance. And conditional random field (CRF) is a popular auxil- iary module for SF task as it considers the correla- tions between tags. Thus, several state-of-the-art works combine the autoregressive model and CRF to achieve the competitive performance, which therefore are set as our baseline methods. However, for SF task, we argue that identifying token dependencies among slot chunk is enough, and it is unnecessary to model …

Content truncated to 500 words