Exploiting Acoustic and Syntactic Features for Prosody Labeling in a Maximum Entropy Framework

Proceedings of NAACL HLT 2007, pages 1–8, Rochester, NY, April 2007. c⃝2007 Association for Computational Linguistics Exploiting acoustic and syntactic features for prosody labeling in a maximum entropy framework Vivek Rangarajan, Shrikanth Narayanan Speech Analysis and Interpretation Laboratory University of Southern California Viterbi School of Electrical Engineering vrangara@usc.edu,shri@sipi.usc.edu Srinivas Bangalore AT&T Research Labs 180 Park Avenue Florham Park, NJ 07932, U.S.A. srini@research.att.com Abstract In this paper we describe an automatic prosody labeling framework that exploits both language and speech information. We model the syntactic-prosodic informa- tion with a maximum entropy model that achieves an accuracy of 85.2% and 91.5% for pitch accent and boundary tone la- beling on the Boston University Radio News corpus. We model the acoustic- prosodic stream with two different mod- els, one a maximum entropy model and the other a traditional HMM. We finally couple the syntactic-prosodic and acoustic- prosodic components to achieve signifi- cantly improved pitch accent and bound- ary tone classification accuracies of 86.0% and 93.1% respectively. Similar experimen- tal results are also reported on Boston Di- rections corpus. 1 Introduction Prosody refers to intonation, rhythm and lexical stress patterns of spoken language that convey lin- guistic and paralinguistic information such as em- phasis, intent, attitude and emotion of a speaker. Prosodic information associated with a unit of speech, say, syllable, word, phrase or clause, influ- ence all the segments of the unit in an utterance. In this sense they are also referred to as suprasegmen- tals (Lehiste, 1970). Prosody in general is highly dependent on individual speaker style, gender, di- alect and other phonological factors. The difficulty in reliably characterizing suprasegmental information present in speech has resulted in symbolic and para- meteric prosody labeling standards like ToBI (Tones and Break Indices) (Silverman et al., 1992) and Tilt model (Taylor, 1998) respectively. Prosody in spoken language can be characterized through acoustic features or lexical features or both. Acoustic correlates of duration, intensity and pitch, like syllable nuclei duration, short time energy and fundamental frequency (f0) are some acoustic fea- tures that are perceived to confer prosodic promi- nence or stress in English. Lexical features like parts- of-speech, syllable nuclei identity, syllable stress of neighboring words have also demonstrated high de- gree of discriminatory evidence in prosody detection tasks. The interplay between acoustic and lexical fea- tures in characterizing prosodic events has been suc- cessfully exploited in text-to-speech synthesis (Bu- lyko and Ostendorf, 2001; Ma et al., 2003), speech recognition (Hasegawa-Johnson et al., 2005) and speech understanding (Wightman and Ostendorf, 1994). Text-to-speech synthesis relies on lexical fea- tures derived predominantly from the input text to synthesize natural sounding speech with appropri- ate prosody. In contrast, output of a typical auto- matic speech recognition (ASR) system is noisy and hence, the acoustic features are more useful in pre- dicting prosody than the hypothesized lexical tran- script which may be erroneous. Speech understand- ing systems model both the lexical and acoustic fea- tures at the output of an ASR to improve natural language understanding. Another source of …

Content truncated to 500 words