Dialogue State Tracking with a Language Model using Schema-Driven Prompting

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4937–4949 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 4937 Dialogue State Tracking with a Language Model using Schema-Driven Prompting Chia-Hsuan Lee University of Washington chiahlee@uw.edu Hao Cheng Microsoft Research chehao@microsoft.com Mari Ostendorf University of Washington ostendor@uw.edu Abstract Task-oriented conversational systems often use dialogue state tracking to represent the user’s intentions, which involves filling in values of pre-defined slots. Many approaches have been proposed, often using task-specific archi- tectures with special-purpose classifiers. Re- cently, good results have been obtained us- ing more general architectures based on pre- trained language models. Here, we introduce a new variation of the language modeling ap- proach that uses schema-driven prompting to provide task-aware history encoding that is used for both categorical and non-categorical slots. We further improve performance by augmenting the prompting with schema de- scriptions, a naturally occurring source of in- domain knowledge. Our purely generative sys- tem achieves state-of-the-art performance on MultiWOZ 2.2 and achieves competitive per- formance on two other benchmarks: Multi- WOZ 2.1 and M2M. The data and code will be available at https://github.com/ chiahsuan156/DST-as-Prompting. 1 Introduction In task-oriented dialogues, systems communicate with users through natural language to accomplish a wide range of tasks, such as food ordering, tech sup- port, restaurant/hotel/travel booking, etc. The back- bone module of a typical system is dialogue state tracking (DST), where the user goal is inferred from the dialogue history (Henderson et al., 2014; Shah et al., 2018; Budzianowski et al., 2018). User goals are represented in terms of values of pre-defined slots associated with a schema determined by the information needed to execute task-specific queries to the backend. In other words, user goals are ex- tracted progressively via slot filling based on the schema throughout the conversation. In this paper, we focus on multi-domain DST where the dialogue state is encoded as a list of triplets in the form of (domain, slot, value), e.g. (“restaurant”, “area”, “centre”). There are two broad paradigms of DST models, classification-based and generation-based models, where the major difference is how the slot value is inferred. In classification-based models (Ye et al., 2021; Chen et al., 2020), the prediction of a slot value is restricted to a fixed set for each slot, and non-categorical slots are constrained to values ob- served in the training data. In contrast, generation- based models (Wu et al., 2019; Kim et al., 2020) decode slot values sequentially (token by token) based on the dialogue context, with the potential of recovering unseen values. Recently, generation- based DST built on large-scale pretrained neural language models (LM) achieve strong results with- out relying on domain-specific modules. Among them, the autoregressive model (Peng et al., 2020a; Hosseini-Asl et al., 2020) uses a uni-directional encoder whereas the sequence-to-sequence model (Lin et al., 2020a; Heck et al., 2020) represents the dialogue context using a bi-directional encoder. In this study, we follow a generation-based DST approach using a pre-trained sequence-to-sequence model, but with the new strategy of …

Content truncated to 500 words