Modeling Conversational Speech for Speech Recognition

Modeling Conversational Speech for Speech Recognition Marie Meteer and Rukmini Iyer BBN Systems & Technologies 70 Fawcett St. Cambridge MA 02138 mmeteer@bbn.com, fiyer@bbn.com Abstract In language modeling for speech recognition the goal is to constrain the search of the speech recognizer by providing a model which can, given a context, indicate what the next most likely word will be. In this paper, we explore how the addition of information to the text, in particular part of speech and dysfluency annotations, can be used to,build more complex language models. In particular, we ask two questions. First, in conversational speech, where there is a less clear notion of "sentence" than in written text, does segmenting the text into linguistically or semantically based units contribute to a better language model than merely segmenting based on broad acoustic information, such as pauses. Second, is the sentence itself a good unit to be modeling, or should we look at smaller units, for example, dividing a sentence into a "given" and "new" portion and segmenting out acknowledgments and replies. To answer these questions, we present a variety of kinds of analysis, from vocabulary distributions to perplexities on language models. The next step will be modeling conversations and incorporating those models into a speech recognizer. 1 Introduction In language modeling for speech recognition the goal is to constrain the search of the speech recognizer by providing a model which can, given a context, indicate what the next most likely word will be. Currently, the field is predominated by the use of n-grams, in particular bi-grams and tri-grams. One advantage of this type of model is that only the text itself is needed to create the model. In the case where what is being modeled is written text in a style for which millions of words of text are available, such as the WSJ corpus, this kind of modeling is effective. However, this is rarely the case since the ultimate goal of speech recognition is to model extemporaneous spoken language. The interesting problem in language modeling is how to bring generalizations above the level of the words themselves to the text. One approach is to annotate text, either by hand or semi-automatically, to bring additional information to the text. Another is to develop algorithms that use rules or heuristics that operate over the corpus, again bringing additional information. It is this additional information that can help create generalizations over the text and contribute to a model which can go beyond the training corpus. In this paper, we describe annotations to the Switchboard corpus which add part of speech and dysfluency markings and our latest work using those annotations to create a level of generalization on top of the annotation to capture the structure of conversational speech. In particular, we ask two questions. First, in conversational speech, where there is a less clear notion of "sentence" than in written text, does segmenting the text into linguistically or semantically based units contribute to a better language model than merely segmenting …

Content truncated to 500 words