CM-Net: A Novel Collaborative Memory Network for Spoken Language Understanding

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1051–1060, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1051 CM-Net: A Novel Collaborative Memory Network for Spoken Language Understanding Yijin Liu1∗, Fandong Meng2, Jinchao Zhang2, Jie Zhou2, Yufeng Chen1 and Jinan Xu1† 1Beijing Jiaotong University, China 2Pattern Recognition Center, WeChat AI, Tencent Inc, China adaxry@gmail.com {fandongmeng, dayerzhang, withtomzhou}@tencent.com {chenyf, jaxu}@bjtu.edu.cn Abstract Spoken Language Understanding (SLU) mainly involves two tasks, intent detection and slot filling, which are generally modeled jointly in existing works. However, most existing models fail to fully utilize co- occurrence relations between slots and intents, which restricts their potential performance. To address this issue, in this paper we propose a novel Collaborative Memory Network (CM-Net) based on the well-designed block, named CM-block. The CM-block firstly cap- tures slot-specific and intent-specific features from memories in a collaborative manner, and then uses these enriched features to enhance local context representations, based on which the sequential information flow leads to more specific (slot and intent) global utterance representations. Through stacking multiple CM-blocks, our CM-Net is able to alternately perform information exchange among specific memories, local contexts and the global utterance, and thus incrementally enriches each other. We evaluate the CM-Net on two standard benchmarks (ATIS and SNIPS) and a self-collected corpus (CAIS). Experimental results show that the CM-Net achieves the state-of-the-art results on the ATIS and SNIPS in most of criteria, and significantly outperforms the baseline models on the CAIS. Additionally, we make the CAIS dataset publicly available for the research community 1. 1 Introduction Spoken Language Understanding (SLU) is a core component in dialogue systems. It typically aims to identify the intent and semantic constituents ∗ This work was done when Yijin Liu was interning at Pattern Recognition Center, WeChat AI, Tencent Inc, China †Jinan Xu is the corresponding author of the paper. 1Code is available at: https://github.com/Adaxry/CM- Net. playlist_owner artist playlist music_item sort served_dish time_range temperature � ����������� ��������������� AddToPlaylist PlayMusic BookRestaurant GetWeather � Figure 1: Statistical association of slot tags (on the left) and intent labels (on the right) in the SNIPS, where col- ors indicate different intents and thicknesses of lines indicate proportions. for a given utterance, which are referred as in- tent detection and slot filling, respectively. Past years have witnessed rapid developments in di- verse deep learning models (Haffner et al., 2003; Sarikaya et al., 2011) for SLU. To take full ad- vantage of supervised signals of slots and intents, and share knowledge between them, most of ex- isting works apply joint models that mainly based on CNNs (Xu and Sarikaya, 2013; Gupta et al., 2019), RNNs (Guo et al., 2014a; Liu and Lane, 2016), and asynchronous bi-model (Wang et al., 2018). Generally, these joint models encode words convolutionally or sequentially, and then aggre- gate hidden states into a utterance-level represen- tation for the intent prediction, without interac- tions between representations of slots and intents. Intuitively, slots and …

Content truncated to 500 words