Robust Dialogue State Tracking with Weak Supervision and Sparse Data

Robust Dialogue State Tracking with Weak Supervision and Sparse Data Michael Heck, Nurul Lubis, Carel van Niekerk, Shutong Feng, Christian Geishauser, Hsien-Chin Lin, Milica Gaˇsi´c Heinrich Heine University D¨usseldorf, Germany {heckmi,lubis,niekerk,fengs,geishaus,linh,gasic}@hhu.de Abstract Generalizing dialogue state tracking (DST) to new data is especially challenging due to the strong reliance on abundant and fine-grained supervision during training. Sample sparsity, distributional shift, and the occurrence of new concepts and topics frequently lead to severe performance degradation during inference. In this paper we propose a training strategy to build extractive DST models without the need for fine-grained manual span labels. Two novel input-level dropout methods mitigate the negative impact of sample sparsity. We propose a new model architecture with a uni- fied encoder that supports value as well as slot independence by leveraging the attention mechanism. We combine the strengths of tri- ple copy strategy DST and value matching to benefit from complementary predictions with- out violating the principle of ontology inde- pendence. Our experiments demonstrate that an extractive DST model can be trained with- out manual span labels. Our architecture and training strategies improve robustness towards sample sparsity, new concepts, and topics, leading to state-of-the-art performance on a range of benchmarks. We further highlight our model’s ability to effectively learn from non-dialogue data. 1 Introduction Generalization and robustness are among the key requirements for naturalistic conversational abil- ities of task-oriented dialogue systems (Edlund et al., 2008). In a dialogue system, dialogue state tracking (DST) solves the task of extracting mean- ing and intent from the user input, and keeps track of the user’s goal over the continuation of a con- versation as part of a dialogue state (DS) (Young et al., 2010). A recommendation and booking sys- tem for places, for instance, needs to gather user preferences in terms of budget, location, and so forth. Concepts like these are assembled in an ontology on levels of domain (e.g., restaurant or hotel), slot (e.g., price or location), and value (e.g., ‘‘expensive’’ or ‘‘south’’). Accurate DST is vital to a robust dialogue system, as the system’s future actions depend on the conversation’s cur- rent estimated state. However, generalizing DST to new data and domains is especially challeng- ing. The reason is the strong reliance on super- vised training. Virtually all top-performing DST methods ei- ther entirely or partially extract values directly from context (Ni et al., 2021). However, train- ing these models robustly is a demanding task. Extractive methods usually rely on fine-grained labels on word level indicating the precise loca- tions of value mentions. Given the richness of human language and the ability to express the same canonical value in many different ways, producing such labels is challenging and very costly, and it is no surprise that datasets of such kind are rare (Zhang et al., 2020b; Deriu et al., 2021). Reliance on detailed labels has another downside; datasets are usually severely limited in size. This in turn leads to the problem of sample sparsity, which increases the risk for models to over-fit to …

Content truncated to 500 words