Using Conditional Random Fields to Predict Pitch Accents in Conversational Speech

Using Conditional Random Fields to Predict Pitch Accents in Conversational Speech Michelle L. Gregory Linguistics Department University at Buffalo Buffalo, NY 14260 mgregory@buffalo.edu Yasemin Altun Department of Computer Science Brown University Providence, RI 02912 altun@cs.brown.edu Abstract The detection of prosodic characteristics is an im- portant aspect of both speech synthesis and speech recognition. Correct placement of pitch accents aids in more natural sounding speech, while automatic detection of accents can contribute to better word- level recognition and better textual understanding. In this paper we investigate probabilistic, contex- tual, and phonological factors that influence pitch accent placement in natural, conversational speech in a sequence labeling setting. We introduce Con- ditional Random Fields (CRFs) to pitch accent pre- diction task in order to incorporate these factors ef- ficiently in a sequence model. We demonstrate the usefulness and the incremental effect of these fac- tors in a sequence model by performing experiments on hand labeled data from the Switchboard Corpus. Our model outperforms the baseline and previous models of pitch accent prediction on the Switch- board Corpus. 1 Introduction The suprasegmental features of speech relay critical information in conversation. Yet, one of the ma- jor roadblocks to natural sounding speech synthe- sis has been the identification and implementation of prosodic characteristics. The difficulty with this task lies in the fact that prosodic cues are never ab- solute; they are relative to individual speakers, gen- der, dialect, discourse context, local context, phono- logical environment, and many other factors. This is especially true of pitch accent, the acoustic cues that make one word more prominent than others in an utterance. For example, a word with a fundamen- tal frequency (f0) of 120 Hz would likely be quite prominent in a male speaker, but not for a typical fe- male speaker. Likewise, the accent on the utterance ”Jon’s leaving.” is critical in determining whether it is the answer to the question ”Who is leaving?” (”JON’s leaving.”) or ”What is Jon doing?” (”Jon’s LEAVING.”). Accurate pitch accent prediction lies in the successful combination of as many of the con- textual variables as possible. Syntactic information such as part of speech has proven to be a success- ful predictor of accentuation (Hirschberg, 1993; Pan and Hirschberg, 2001). In general, function words are not accented, while content words are. Vari- ous measures of a word’s informativeness, such as the information content (IC) of a word (Pan and McKeown, 1999) and its collocational strength in a given context (Pan and Hirschberg, 2001) have also proven to be useful models of pitch accent. How- ever, in open topic conversational speech, accent is very unpredictable. Part of speech and the infor- mativeness of a word do not capture all aspects of accentuation, as we see in this example taken from Switchboard, where a function word gets accented (accented words are in uppercase): I, I have STRONG OBJECTIONS to THAT. Accent is also influenced by aspects of rhythm and timing. The length of words, in both number of phones and normalized duration, affect …

Content truncated to 500 words