Zero-shot Generalization in Dialog State Tracking through Generative Question Answering

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 1063–1074 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 1063 Zero-shot Generalization in Dialog State Tracking through Generative Question Answering Shuyang Li,1, 2 Jin Cao,1 Mukund Sridhar,1 Henghui Zhu,3 Shang-Wen Li,3 Wael Hamza,1 Julian McAuley2 1Amazon Alexa AI, 2UC San Diego, 3AWS AI {shl008, jmcauley}@eng.ucsd.edu {jincao, harakere, henghui, shangwel, waelhamz}@amazon.com Abstract Dialog State Tracking (DST), an integral part of modern dialog systems, aims to track user preferences and constraints (slots) in task- oriented dialogs. In real-world settings with constantly changing services, DST systems must generalize to new domains and unseen slot types. Existing methods for DST do not generalize well to new slot names and many require known ontologies of slot types and values for inference. We introduce a novel ontology-free framework that supports natural language queries for unseen constraints and slots in multi-domain task-oriented dialogs. Our approach is based on generative question- answering using a conditional language model pre-trained on substantive English sentences. Our model improves joint goal accuracy in zero-shot domain adaptation settings by up to 9% (absolute) over the previous state-of-the- art on the MultiWOZ 2.1 dataset. 1 Introduction Dialog agents are gaining increasing prominence in daily life. These systems aim to assist users via natural language conversations, taking the form of digital assistants who help accomplish everyday tasks by interfacing with connected devices and services. A key component to understanding and enabling these task-oriented dialogs is Dialog State Tracking (DST): extracting user intent and goals from conversations via filling in belief slots (Lemon et al., 2006; Wang and Lemon, 2013). Assistive and recommendation use-cases for dialog agents in production settings are particularly challenging due to constantly changing services and applications with which they interface. Traditional DST systems have achieved high ac- curacy when presented with a known ontology of slot types and valid values (Chen et al., 2020). In a real-world setting, however, a DST model must Figure 1: Based on a dialog history, a natural language questions are provided to our model to query a user’s requirements and preferences (dialog state). generalize to new slot values (e.g. new entities that are not present at training time) and new slot types (e.g. requirements regarding a new application). Recent work has sought to address these issues by posing DST as a reading comprehension or ques- tion answering (QA) task (Gao et al., 2019)—such models predict each slot value independently at any given turn and can theoretically be queried for new slots at inference time. Some approaches toward DST as QA learn em- bedding vectors for each slot and/or domain word (Wu et al., 2019), but this is not robust to unseen slots whose specific names (e.g. ‘Internet Access’) may be totally unlike those in the training set. Gao et al. (2020) attempt to remedy this by posing a natural language question for each slot, but their hybrid span-extraction and classification-based sys- tem nonetheless requires access to the full ontology …

Content truncated to 500 words