Prosody/Parse Scoring and Its Application in ATIS

PROSODY/PARSE SCORING AND ITS APPLICATION iN ATIS N. M. Veilleuz M. Ostendorf Electrical, Computer and Systems Engineering Boston University, Boston, MA 02215 ABSTRACT Prosodic patterns provide important cues for resolving syn- tactic ambiguity, and might he used to improve the accu- racy of automatic speech understanding. With this goal, we propose a method of scoring syntactic parses in terms of ob- served prosodic cues, which can be used in ranking sentence hypotheses and associated parses. Specifically, the score is the probability of acoustic features of a hypothesized word sequence given an associated syntactic parse, based on acous- tic and "language" (prosody/syntax) models that represent probabilities in terms of abstract prosodic labeis. This work reports initial efforts aimed at extending the algorithm to spontaneous speech, specifically the ATIS task, where the prosody/parse score is shown to improve the average rank of the correct sentence hypothesis. 1. INTRODUCTION Human listeners bring several sources of information to bear in interpreting an utterance, including syn- tax, semantics, discourse, pragmatics and prosodic cues. Prosody, in particular, provides information about syn- tactic structure (via prosodic constituent structure) and information focus (via phrasal prominence), and is en- coded in the acoustic signal in terms of timing, energy and intonation patterns. Since computer knowledge rep- resentations are not as sophisticated as human knowl- edge, utterances that are straightforward for a human to interpret may be "ambiguous" to an automatic speech understanding system. For this reason, it is useful to include as many knowledge sources as possible in auto- matic speech understanding, and prosody is currently an untapped resource. In fact, some syntactic ambiguities can be resolved by listeners from prosody alone [1]. One way to incorporate prosody in speech understand- ing is to score the expected prosodic structure for each candidate sentence hypothesis and syntactic parse in re- lation to the observed prosodic structure. In a speech understanding system where multiple sentence hypothe- ses are passed from recognition to natural language pro- cessing, the prosody/parse score could be used to rank hypotheses and associated parses, directly or in combina- tion with other scores. The parse scoring approach was proposed in previous work [2], where automatically de- tected prosodic phrase breaks were scored either in terms of their correlation with prosodic structure predicted from parse information or in terms of their likelihood according to a probabilistic prosody/syntax model. Re- cently, the parse scoring approach was reformulated [3] to avoid explicit recognition of prosodic patterns, which is a sub-optimal intermediate decision. Specifically, the new score is the probability of a hypothesized word se- quence and associated syntactic parse given acoustic fea- tures, where both an acoustic model and a "language" (prosody/syntax) model are used to represent the proba- bility of utterance, analogous to speech recognition tech- niques. The parse scoring formalism was also extended to incorporate phrasal prominence information, in ad- dition to phrase breaks. In previous work, we demon- strated the feasibility of using parse scoring to find the correct interpretation in a corpus of professionally read ambiguous sentences. …

Content truncated to 500 words