Improving State-of-the-Art Continuous Speech Recognition Systems Using the N-Best Paradigm with Neural Networks

Improving State-of-the-Art Continuous Speech Recognition Systems Using the N-Best Paradigm with Neural Networks S. Austin, G. Zavaliagkos t, J. Makhoul, and R. Schwartz BBN Systems and Technologies, Cambridge, MA 02138 tNortheastern University, Boston, MA 02115 ABSTRACT In an effort to advance the state of the art in continuous speech recognition employing hidden Markov models (HMM), Segmental Neural Nets (SNN) were introduced recently to ameliorate the well- known limitations of HMMs, namely, the conditional-independence limitation and the relative difficulty with which HMMs can handle segmental features. We describe a hybrid SNN/I-IMM system that combines the speed and performance of our HMM system with the segmental modeling capabilities of SNNs. The integration of the two acoustic modeling techniques is achieved successfully via the N-best rescoring paradigm. The N-best lists are used not only for recognition, but also during training. This discriminative training using N-best is demonstrated to improve performance. When tested on the DARPA Resource Management speaker-independent corpus, the hybrid SNN/HMM system decreases the error by about 20% compared to the state-of-the-art HMM system. INTRODUCTION In February 1991, we introduced at the DARPA Speech and Natural Language Workshop the concept of a Segmen- tal Neural Net (SNN) for phonetic modeling in continuous speech recognition [1]. The SNN was introduced to over- come some of the well-known limitations of hidden Markov models (HMM) which now represent the state of the art in continuous speech recognition (CSR). Two such limitations are (i) the conditional-independence assumption, which pre- vents a HMM from taking full advantage of the correlation that exists among the frames of a phonetic segment, and (ii) the awkwardness with which segmental features (such as duration) can be incorporated into HMM systems. We developed the concept of SNN specifically to overcome the two HMM limitations just mentioned for phonetic modeling in speech. However, neural nets are known to require a large amount of computation, especially for training. Also, there is no known efficient search technique for finding the best scoring segmentation with neural nets in continuous speech. Therefore, we have developed a hybrid SNN/HMM system that is designed to take full advantage of the good prop- erties of both methods: the phonetic modeling properties of SNNs and the good computational properties of HMMs. The two methods are integrated through the use of the N- best paradigm, which was developed in conjunction with the BYBLOS system at BBN [7,6]. A year ago, we presented very preliminary results using our hybrid system on the speaker-dependent portion of the DARPA Resource Management Corpus [1]. Also, the train- ing of the neural net was performed only on the correct tran- scription of the utterances. In this paper, we describe the per- formance of the hybrid system on the speaker-independent portion of the Resource Management corpus, using discrim- inative training on the whole N-best list. Below, we give a description of the SNN, the integration of the SNN with the HMM models using the N-best paradigm, the training of the hybrid SNN/I-IMM system using the whole N-best list, …

Content truncated to 500 words