Discriminative state tracking for spoken dialog systems

Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 466–475, Sofia, Bulgaria, August 4-9 2013. c⃝2013 Association for Computational Linguistics Discriminative state tracking for spoken dialog systems Angeliki Metallinou1∗, Dan Bohus2, and Jason D. Williams2 1University of Southern California, Los Angeles, CA, USA 2Microsoft Research, Redmond, WA, USA metallin@usc.edu dbohus@microsoft.com jason.williams@microsoft.com Abstract In spoken dialog systems, statistical state tracking aims to improve robustness to speech recognition errors by tracking a posterior distribution over hidden dialog states. Current approaches based on gener- ative or discriminative models have differ- ent but important shortcomings that limit their accuracy. In this paper we discuss these limitations and introduce a new ap- proach for discriminative state tracking that overcomes them by leveraging the problem structure. An offline evaluation with dialog data collected from real users shows improvements in both state track- ing accuracy and the quality of the pos- terior probabilities. Features that encode speech recognition error patterns are par- ticularly helpful, and training requires rel- atively few dialogs. 1 Introduction Spoken dialog systems interact with users via nat- ural language to help them achieve a goal. As the interaction progresses, the dialog manager main- tains a representation of the state of the dialog in a process called dialog state tracking. For ex- ample, in a bus schedule information system, the dialog state might indicate the user’s desired bus route, origin, and destination. Dialog state track- ing is difficult because automatic speech recog- nition (ASR) and spoken language understand- ing (SLU) errors are common, and can cause the system to misunderstand the user’s needs. At the same time, state tracking is crucial because the system relies on the estimated dialog state to choose actions – for example, which bus schedule information to present to the user. The dialog state tracking problem can be for- malized as follows (Figure 1). Each system turn in the dialog is one datapoint. For each datapoint, the input consists of three items: a set of K fea- tures that describes the current dialog context, G dialog state hypotheses, and for each dialog state hypothesis, M features that describe that dialog state hypothesis. The task is to assign a probabil- ity distribution over the G dialog state hypotheses, plus a meta-hypothesis which indicates that none of the G hypotheses is correct. Note that G varies across turns (datapoints) – for example, in the first turn of Figure 1, G = 3, and in the second and third turns G = 5. Also note that the dialog state tracker is not predicting the contents of the dialog state hypotheses; the di- alog state hypotheses contents are given by some external process, and the task is to predict a proba- bility distribution over them, where the probability assigned to a hypothesis indicates the probability that it is correct. It is a requirement that the G hypotheses are disjoint; with the special “every- thing else” meta-hypothesis, exactly one hypoth- esis is correct by construction. After the dialog state tracker has …

Content truncated to 500 words