Uncertainty Measures in Neural Belief Tracking and the Effects on Dialogue Policy Performance

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7901–7914 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 7901 Uncertainty Measures in Neural Belief Tracking and the Effects on Dialogue Policy Performance Carel van Niekerk1, Andrey Malinin2, Christian Geishauser1, Michael Heck1 Hsien-chin Lin1, Nurul Lubis1, Shutong Feng1 and Milica Gaši´c1 1Heinrich Heine Universität Düsseldorf, Düsseldorf, Germany 2Yandex Research and HSE University, Moscow, Russia 1{niekerk,geishaus,heckmi,linh,lubis,shutong.feng,gasic}@hhu.de 2am969@yandex-team.ru Abstract The ability to identify and resolve uncertainty is crucial for the robustness of a dialogue sys- tem. Indeed, this has been confirmed em- pirically on systems that utilise Bayesian ap- proaches to dialogue belief tracking. How- ever, such systems consider only confidence estimates and have difficulty scaling to more complex settings. Neural dialogue systems, on the other hand, rarely take uncertainties into account. They are therefore overconfident in their decisions and less robust. Moreover, the performance of the tracking task is often evaluated in isolation, without consideration of its effect on the downstream policy optimisa- tion. We propose the use of different uncer- tainty measures in neural belief tracking. The effects of these measures on the downstream task of policy optimisation are evaluated by adding selected measures of uncertainty to the feature space of the policy and training poli- cies through interaction with a user simulator. Both human and simulated user results show that incorporating these measures leads to im- provements both of the performance and of the robustness of the downstream dialogue policy. This highlights the importance of developing neural dialogue belief trackers that take uncer- tainty into account. 1 Introduction In task-oriented dialogue, the system aims to assist the user in obtaining information. This is achieved through a series of interactions between the user and the system. As the conversation progresses, it is the role of the dialogue state tracking module to track the state of the conversation. For example, in a restaurant recommendation system, the state would include the information about the cuisine of the desired restaurant, its area as well as the price range that the user has in mind. It is crucial that this state contains all information necessary for the dialogue policy to make an informed decision for the next action (Young et al., 2007). Policy training optimises decision making in order to complete dialogues successfully. It has been proposed within the partially observ- able Markov decision process (POMDP) approach to dialogue modelling to track the distribution over all possible dialogue states, the belief state, instead of a single most-likely candidate. This approach successfully integrates uncertainty to achieve ro- bustness (Williams and Young, 2007; Thomson and Young, 2010; Young et al., 2016, 2007). However, such systems do not scale well to complex multi- domain dialogues. On the other hand, discrimina- tive neural approaches to dialogue tracking achieve state-of-the-art performance in the state tracking task. Nevertheless, the state-of-the-art goal ac- curacy on the popular MultiWOZ (Budzianowski et al., 2018) multi-domain benchmark is currently only at 60% (Heck et al., 2020; Li et al., 2020a). In …

Content truncated to 500 words