Combining Acoustic and Pragmatic Features to Predict Recognition Performance in Spoken Dialogue Systems

Combining Acoustic and Pragmatic Features to Predict Recognition Performance in Spoken Dialogue Systems Malte Gabsdil Department of Computational Linguistics Saarland University Germany gabsdil@coli.uni-sb.de Oliver Lemon School of Informatics Edinburgh University Scotland olemon@inf.ed.ac.uk Abstract We use machine learners trained on a combina- tion of acoustic confidence and pragmatic plausi- bility features computed from dialogue context to predict the accuracy of incoming n-best recogni- tion hypotheses to a spoken dialogue system. Our best results show a 25% weighted f-score improve- ment over a baseline system that implements a “grammar-switching” approach to context-sensitive speech recognition. 1 Introduction A crucial problem in the design of spoken dialogue systems is to decide for incoming recognition hy- potheses whether a system should accept (consider correctly recognized), reject (assume misrecogni- tion), or ignore (classify as noise or speech not di- rected to the system) them. In addition, a more so- phisticated dialogue system might decide whether to clarify or confirm certain hypotheses. Obviously, incorrect decisions at this point can have serious negative effects on system usability and user satisfaction. On the one hand, accepting misrecognized hypotheses leads to misunderstand- ings and unintended system behaviors which are usually difficult to recover from. On the other hand, users might get frustrated with a system that be- haves too cautiously and rejects or ignores too many utterances. Thus an important feature in dialogue system engineering is the tradeoff between avoiding task failure (due to misrecognitions) and promoting overall dialogue efficiency, flow, and naturalness. In this paper, we investigate the use of machine learners trained on a combination of acoustic confi- dence and pragmatic plausibility features (i.e. com- puted from dialogue context) to predict the qual- ity of incoming n-best recognition hypotheses to a spoken dialogue system. These predictions are then used to select a “best” hypothesis and to de- cide on appropriate system reactions. We evalu- ate this approach in comparison with a baseline system that combines fixed recognition confidence rejection thresholds with dialogue-state dependent recognition grammars (Lemon, 2004). The paper is organized as follows. After a short relation to previous work, Section 3 introduces the WITAS multimodal dialogue system, which we use to collect data (Section 4) and to derive baseline re- sults (Section 5). Section 6 describes our learning experiments for classifying and selecting from n- best recognition hypotheses and Section 7 reports our results. 2 Relation to Previous Work (Litman et al., 2000) use acoustic-prosodic infor- mation extracted from speech waveforms, together with information derived from their speech recog- nizer, to automatically predict misrecognized turns in a corpus of train-timetable information dialogues. In our experiments, we also use recognizer con- fidence scores and a limited number of acoustic- prosodic features (e.g. amplitude in the speech sig- nal) for hypothesis classification. (Walker et al., 2000) use a combination of features from the speech recognizer, natural language understanding, and di- alogue manager/discourse history to classify hy- potheses as correct, partially correct, or misrecog- nized. Our work is related to these experiments in that we also combine confidence scores and …

Content truncated to 500 words