N-Best Rescoring Based on Pitch-accent Patterns

Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pages 732–741, Portland, Oregon, June 19-24, 2011. c⃝2011 Association for Computational Linguistics N-Best Rescoring Based on Pitch-accent Patterns Je Hun Jeon1 Wen Wang2 Yang Liu1 1Department of Computer Science, The University of Texas at Dallas, USA 2Speech Technology and Research Laboratory, SRI International, USA {jhjeon,yangl}@hlt.utdallas.edu, wwang@speech.sri.com Abstract In this paper, we adopt an n-best rescoring scheme using pitch-accent patterns to improve automatic speech recognition (ASR) perfor- mance. The pitch-accent model is decoupled from the main ASR system, thus allowing us to develop it independently. N-best hypothe- ses from recognizers are rescored by addi- tional scores that measure the correlation of the pitch-accent patterns between the acoustic signal and lexical cues. To test the robustness of our algorithm, we use two different data sets and recognition setups: the first one is En- glish radio news data that has pitch accent la- bels, but the recognizer is trained from a small amount of data and has high error rate; the sec- ond one is English broadcast news data using a state-of-the-art SRI recognizer. Our experi- mental results demonstrate that our approach is able to reduce word error rate relatively by about 3%. This gain is consistent across the two different tests, showing promising future directions of incorporating prosodic informa- tion to improve speech recognition. 1 Introduction Prosody refers to the suprasegmental features of nat- ural speech, such as rhythm and intonation, since it normally extends over more than one phoneme segment. Speakers use prosody to convey paralin- guistic information such as emphasis, intention, atti- tude, and emotion. Humans listening to speech with natural prosody are able to understand the content with low cognitive load and high accuracy. How- ever, most modern ASR systems only use an acous- tic model and a language model. Acoustic informa- tion in ASR is represented by spectral features that are usually extracted over a window length of a few tens of milliseconds. They miss useful information contained in the prosody of the speech that may help recognition. Recently a lot of research has been done in au- tomatic annotation of prosodic events (Wightman and Ostendorf, 1994; Sridhar et al., 2008; Anan- thakrishnan and Narayanan, 2008; Jeon and Liu, 2009). They used acoustic and lexical-syntactic cues to annotate prosodic events with a variety of machine learning approaches and achieved good performance. There are also many studies us- ing prosodic information for various spoken lan- guage understanding tasks. However, research using prosodic knowledge for speech recognition is still quite limited. In this study, we investigate leverag- ing prosodic information for recognition in an n-best rescoring framework. Previous studies showed that prosodic events, such as pitch-accent, are closely related with acous- tic prosodic cues and lexical structure of utterance. The pitch-accent pattern given acoustic signal is strongly correlated with lexical items, such as syl- lable identity and canonical stress pattern. There- fore as a first study, we focus on pitch-accent in this paper. We develop two separate pitch-accent …

Content truncated to 500 words