Characterizing and Recognizing Spoken Corrections in Human-Computer Dialogue

Characterizing and Recognizing Spoken Corrections in Human-Computer Dialogue Gina-Anne Levow MIT AI Laboratory Room 769, 545 Technology Sq Cambridge, MA 02139 gina@ai.mit.edu Abstract Miscommunication in speech recognition sys- tems is unavoidable, but a detailed character- ization of user corrections will enable speech systems to identify when a correction is taking place and to more accurately recognize the con- tent of correction utterances. In this paper we investigate the adaptations of users when they encounter recognition errors in interactions with a voice-in/voice-out spoken language system. In analyzing more than 300 pairs of original and re- peat correction utterances, matched on speaker and lexical content, we found overall increases in both utterance and pause duration from orig- inal to correction. Interestingly, corrections of misrecognition errors (CME) exhibited signifi- cantly heightened pitch variability, while cor- rections of rejection errors (CRE) showed only a small but significant decrease in pitch minimmn. CME's demonstrated much greater increases in measures of duration and pitch variability than CRE's. These contrasts allow the development of decision trees which distinguish CME's from CRE's and from original inputs at 70-75% ac- curacy based on duration, pitch, and amplitude features. 1 Introduction The frequent recognition errors which plague speech recognition systems present a signifi- cant barrier to widespread acceptance of this technology. The difficulty of correcting sys- tern misrecognitions is directly correlated with user assessments of system quality. The in- creased probability of recognition errors imme- diately after an error compounds this prob- lem. Thus, it becomes crucially important to characterize the differences between origi- nal utterances and user corrections of system recognition failures both in order to recognize when a user attempts a correction, indicating a prior recognition error, and to improve recogni- tion accuracy on these problematic utterances. Analysis of data drawn from a field trial of a telephone-based voice-in/voice-out conversa- tional system demonstrates significant differ- ences between original inputs and corrections in measures of duration, pause, and pitch. These differences in turn aid in tile development of de- cision trees which distinguish between new in- put and user corrections. 2 Overview We begin by describing the framework in which the utterances used for these experiments were collected. We then describe the acoustic fea- tures used in the data analysis and decision tree design and motivate this choice by considering some related work. A report of the results of the descriptive analysis follows and demonstrates the significant differences among original inputs and different types of corrections. We then turn to the issues involved in the actual construction and assessment of decision trees which identify spoken corrections. The best of these classi- fiers achieves 75% success rates. Finally, we conclude with a plan to use these results to improve recognition accuracy on spoken correc- tions and ease recovery from hmnan-computer miscommunication. 3 System, Subjects, and Errors The utterances used in the design and analy- sis of the decision tree classifiers were drawn from approximately 60 hours of user interac- tions in a field trial of the Sun Microsystems SpeechActs system …

Content truncated to 500 words