Integration of Diverse Recognition Methodologies Through Reevaluation of N-Best Sentence Hypotheses

Integration of Diverse Recognition Methodologies Through Reevaluation of N-Best Sentence Hypotheses M. Ostendor~ A. Kannan~ S. Auagin$ O. Kimballt R. Schwartz.]: J.R. Rohlieek~: t Boston University 44 Cummington St. Boston, MA 02215 ABSTRACT This paper describes a general formalism for integrating two or more speech recognition technologies, which could be devel- oped at different research sites using different recognition strate- gies. In this formalism, one system uses the N-best search strat- egy to generate a list of candidate sentences; the list is rescorred by other systems; and the different scores axe combined to opti- mize performance. Specifically, we report on combining the BU system based on stochastic segment models and the BBN sys- tem based on hidden Markov models. In addition to facilitating integration of different systems, the N-best approach results in a large reduction in computation for word recognition using the stochastic segment model INTRODUCTION While most successful systems to date have been based on hidden Markov models (HMMs), there may be utility in combining the HMM approach with some other very differ- ent approach. For example, the research group at Boston University is exploring the use of the Stochastic Segment Model (SSM) [9,11] as an alternative to the HMM. In con- trast to the HMM, the SSM scores a phoneme as a whole entity, allowing a more detailed acoustic representation. In previous work [6], it was demonstrated that the SSM is ef- fective in the task of phoneme recognition, with results on the TIMIT database using context-independent phoneme models that are comparable to context-dependent HMMs. Thus, there is a good possibility that, with the proper use of context, the performance may surpass that of the HMM system. Unfortunately, the computation required for the SSM is considerably greater than that for HMMs, making it impractical to implement the standard optimal dynamic programming search algorithms. In this paper, we introduce a general formalism for inte- grating different speech recognition technologies, which also enables evaluation of word recognition performance with the SSM. In this approach, one recognition system uses the N- best search strategy to provide a list of sentence hypothe- ses. A second system (presumably more complex) is used to rescore these hypotheses, and the scores of the different sys- BBN Inc. 10 Moulton St. Cambridge, MA 02138 terns are combined, giving a new ranking of the sentence hy- potheses. If the errors made by the two systems differ, then combining the two sets of scores would yield an improve- ment in overall performance (either in terms of the percent of correct sentences or the average rank of the correct sen- tence). The N-best formalism offers a means of reducing the computation associated with combining the results of two systems by restricting the search space of the second system. It therefore also provides a lower cost mechanism for evaluating word recognition performance of the SSM by itself, through simply ignoring the scores of the HMM in reranking the sentences. In the following section, we describe the integration method- ology in …

Content truncated to 500 words