A Speech-First Model for Repair Detection and Correction

A SPEECH-FIRST MODEL FOR REPAIR DETECTION AND CORRECTION Christine Nakatani Division of Applied Sciences Harvard University Cambridge, MA 02138 chn@das, harvard, edu Julia Hirschberg 2D-450, AT&T Bell Laboratories 600 Mountain Avenue Murray Hill, NJ 07974-0636 julia@research, att. com Abstract Interpreting fully natural speech is an important goal for spoken language understanding systems. However, while corpus studies have shown that about 10% of spontaneous utterances contain self-corrections, or RE- PAIRS, little is known about the extent to which cues in the speech signal may facilitate repair processing. We identify several cues based on acoustic and prosodic analysis of repairs in a corpus of spontaneous speech, and propose methods for exploiting these cues to detect and correct repairs. We test our acoustic-prosodic cues with other lexical cues to repair identification and find that precision rates of 89-93% and recall of 78-83% can be achieved, depending upon the cues employed, from a prosodically labeled corpus. Introduction Disfluencies in spontaneous speech pose serious prob- lems for spoken language systems. First, a speaker may produce a partial word or FRAGMENT, a string of phonemes that does not form the complete intended word. Some fragments may coincidentally match words actually in the lexicon, such as fly in Exam- ple (1); others will be identified with the acoustically closest item(s) in the lexicon, as in Example (2). 1 (1) What is the earliest fli- flight from Washington to Atlanta leaving on Wednesday September fourth? (2) Actual string: What is the fare fro- on American Airlines fourteen forty three Recognized string: With fare four American Air- lines fourteen forty three Even if all words in a disfluent segment are correctly recognized, failure to detect a disfluency may lead to interpretation errors during subsequent processing, as in Example (3). 1The presence of a word fragment in examples is indicated by the diacritic '-'. Self-corrected portions of the utterance appear in boldface. All examples in this paper are drawn from the ATIS corpus described below. Recognition output shown in Example (2) is from the system described in (Lee et al., 1990). (3) ... Delta leaving Boston seventeen twenty one ar- riving Fort Worth twenty two twenty one forty... Here, 'twenty two twenty one forty' must be interpreted as a flight arrival time; the system must somehow choose among '21:40', '22:21', and '22:40'. Although studies of large speech corpora have found that approximately 10% of spontaneous utter- ances contain disfluencies involving self-correction, or REPAIRS (Hindle, 1983; Shriberg et al., 1992), little is known about how to integrate repair processing with real-time speech recognition. In particular, the speech signal itself has been relatively unexplored as a source of processing cues for the detection and correction of repairs. In this paper, we present results from a study of the acoustic and prosodic characteristics of 334 repair utterances, containing 368 repair instances, from the AROA Air Travel Information System (ATIS) database. Our results are interpreted within our "speech-first" framework for investigating repairs, the REPAIR IN- TERVAL MODEL (RIM). RIM builds upon Labov (1966) and …

Content truncated to 500 words