Neural Dialogue State Tracking with Temporally Expressive Networks

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1570–1579 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 1570 Neural Dialogue State Tracking with Temporally Expressive Networks Junfan Chen BDBC and SKLSDE Beihang University, China chenjf@act.buaa.edu.cn Richong Zhang∗ BDBC and SKLSDE Beihang University, China zhangrc@act.buaa.edu.cn Yongyi Mao School of EECS University of Ottawa, Canada ymao@uottawa.ca Jie Xu School of Computing University of Leeds, United Kingdom j.xu@leeds.ac.uk Abstract Dialogue state tracking (DST) is an important part of a spoken dialogue system. Existing DST models either ignore temporal feature de- pendencies across dialogue turns or fail to ex- plicitly model temporal state dependencies in a dialogue. In this work, we propose Tem- porally Expressive Networks (TEN) to jointly model the two types of temporal dependencies in DST. The TEN model utilizes the power of recurrent networks and probabilistic graph- ical models. Evaluating on standard datasets, TEN is demonstrated to improve the accuracy of turn-level-state prediction and the state ag- gregation. 1 Introduction Spoken dialogue systems (SDS) connect users and computer applications through human-machine conversations. The users can achieve their goals, such as finding a restaurant, by interacting with a task-oriented SDS over multiple dialogue rounds or turns. Dialogue state tracking (DST) is an impor- tant task in SDS and the key function is to maintain the state of the system so as to track the progress of the dialogue. In the context of this work, a state (or aggregated state) is the user’s intention or interest accumulated from the conversation history, and the user’s intention or interest at each turn is referred to as turn-level state. Many neural-network models have been success- fully applied to DST. These models usually solve the DST problem by two approaches, the Implicit Tracking and the Explicit Tracking. As is shown in Figure 1 (a), the Implicit Tracking models (Hen- derson et al., 2014b,c; Mrksic et al., 2015; Ren et al., 2018; Ramadan et al., 2018; Lee et al., 2019) employs recurrent networks to accumulate features extracted from historical system action and user ∗Corresponding author utterance pairs. A classifier is then built upon these accumulated features for state prediction. Although the Implicit Tracking captures temporal feature de- pendencies in recurrent-network cells, the state dependencies are not explicitly modeled. Only considering temporal feature dependencies is in- sufficient for accurate state prediction. This fact has been confirmed via an ablation study in our experiment. Unlike the Implicit Tracking, the Explicit Track- ing approaches, such as NBT (Mrksic et al., 2017) and GLAD (Zhong et al., 2018), model the state dependencies explicitly. From the model structure in Figure 1(b), the Explicit Tracking approaches first build a classifier to predict the turn-level state of each turn and then utilize a state aggregator for state aggregation. Despite achieving remarkable improvements upon the previous models, current Explicit Track- ing models can be further improved in two as- pects. One is that the temporal feature dependen- cies should be considered in model design. The Explicit Tracking models only extract features from …

Content truncated to 500 words