Joint Slot Filling and Intent Detection via Capsule Neural Networks

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5259–5267 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5259 Joint Slot Filling and Intent Detection via Capsule Neural Networks Chenwei Zhang†, Yaliang Li§, Nan Du‡, Wei Fan‡, Philip S. Yu†¶ †University of Illinois at Chicago, Chicago, IL 60607 USA §Alibaba Group, Bellevue, WA 98004 USA ‡Tencent Medical AI Lab, Palo Alto, CA 94301 USA ¶Institute for Data Science, Tsinghua University, Beijing, China {czhang99,psyu}@uic.edu, yaliang.li@alibaba-inc.com, nandu2048@gmail.com, davidwfan@tencent.com Abstract Being able to recognize words as slots and de- tect the intent of an utterance has been a keen issue in natural language understanding. The existing works either treat slot filling and in- tent detection separately in a pipeline man- ner, or adopt joint models which sequentially label slots while summarizing the utterance- level intent without explicitly preserving the hierarchical relationship among words, slots, and intents. To exploit the semantic hierarchy for effective modeling, we propose a capsule- based neural network model which accom- plishes slot filling and intent detection via a dynamic routing-by-agreement schema. A re- routing schema is proposed to further syner- gize the slot filling performance using the in- ferred intent representation. Experiments on two real-world datasets show the effectiveness of our model when compared with other alter- native model architectures, as well as existing natural language understanding services. 1 Introduction With the ever-increasing accuracy in speech recognition and complexity in user-generated ut- terances, it becomes a critical issue for mobile phones or smart speaker devices to understand the natural language in order to give informative re- sponses. Slot filling and intent detection play im- portant roles in Natural Language Understanding (NLU) systems. For example, given an utterance from the user, the slot filling annotates the utter- ance on a word-level, indicating the slot type men- tioned by a certain word such as the slot artist mentioned by the word Sungmin, while the in- tent detection works on the utterance-level to give categorical intent label(s) to the whole utterance. Figure 1 illustrates this idea. To deal with diversely expressed utterances without additional feature engineering, deep neu- ral network based user intent detection models (Hu Word Put Sungmin into my summer playlist Slot O B-artist O B-playlist_owner B-playlist O Intent AddToPlaylist Figure 1: An example of an utterance with BOI format an- notation for slot filling, which indicates the slot of artist, play list owner, and play list name from an utterance with an intent AddToPlaylist. et al., 2009; Xu and Sarikaya, 2013; Zhang et al., 2016; Liu and Lane, 2016; Zhang et al., 2017; Chen et al., 2016; Xia et al., 2018) are proposed to classify user intents given their utterances in the natural language. Currently, the slot filling is usually treated as a sequential labeling task. A neural network such as a recurrent neural network (RNN) or a convolution neural network (CNN) is used to learn context- aware word representations, along with sequence tagging methods such as conditional random field …

Content truncated to 500 words