Syntactic Graph Convolutional Network for Spoken Language Understanding

Proceedings of the 28th International Conference on Computational Linguistics, pages 2728–2738 Barcelona, Spain (Online), December 8-13, 2020 2728 Syntactic Graph Convolutional Network for Spoken Language Understanding Keqing He1∗, Shuyu Lei2, Yushu Yang2, Huixing Jiang2, Zhongyuan Wang2 1Beijing University of Posts and Telecommunications, Beijing, China 2Meituan Group {kqin,leishuyu}@bupt.edu.cn {yangyushu,jianghuixing,wangzhongyuan02}@meituan.com Abstract Slot filling and intent detection are two major tasks for spoken language understanding. In most existing work, these two tasks are built as joint models with multi-task learning with no consid- eration of prior linguistic knowledge. In this paper, we propose a novel joint model that applies a graph convolutional network over dependency trees to integrate the syntactic structure for learn- ing slot filling and intent detection jointly. Experimental results show that our proposed model achieves state-of-the-art performance on two public benchmark datasets and outperforms exist- ing work. At last, we apply the BERT model to further improve the performance on both slot filling and intent detection. 1 Introduction Spoken Language Understanding (SLU) plays a vital role in a task-oriented dialogue system. Slot filling and intent detection (Tur and De Mori, 2011) are two major tasks for SLU as shown in Figure 1(a). Slot filling aims to obtain the semantic structure for the utterance. Meanwhile, intent detection annotates the categorical intent of the utterance. In typical pipeline methods, slot filling and intent detection are built separately. Slot filling is imple- mented as a standard sequence labeling task (Yao et al., 2014) and intent detection is built as a classifi- cation task (Lai et al., 2015), respectively. Essentially, slot filling and intent detection impact mutually. Therefore, more prior work (Hakkani-T¨ur et al., 2016; Liu and Lane, 2016; Goo et al., 2018; Li et al., 2018; Wang et al., 2018; Zhang et al., 2018a; E et al., 2019; Qin et al., 2019) implement two afore- mentioned tasks jointly as multi-task learning and achieve more promising results than those pipeline methods. However, most prior work utilize sequential model, such as recurrent neural network, to ac- cumulate the contextual representation for each word to implement SLU with no consideration of prior linguistic knowledge. Intuitively, slot filling and intent detection rely on indicative contextual words for disambiguation and suffer from the degradation on wide contexts. Syntactic dependency parse tree as shown in Figure 1(b), which provides linguistic dependency relation among words, has been shown generally beneficial in various NLP tasks such as machine reading comprehension (Zhang et al., 2019), neural machine translation (Chen et al., 2018) and relation extraction (Zhang et al., 2018b). The major reasons are that the dependency parse tree can capture long-range relations between words and contain implicit clues for disambiguation. To access a better SLU, we emphasize that SLU model should utilize the dependency representation as prior linguistic knowledge. In this paper, we propose a joint SLU model that applies a Graph Convo- lutional Network (GCN) over dependency trees to integrate the syntactic structure for joint learning slot filling and intent detection, where the GCN can pool information over arbitrary dependency structures …

Content truncated to 500 words