Continuous Speech Recognition Using Segmental Neural Nets

Continuous Speech Recognition Using Segmental Neural Nets S. Austin'l', J. Makhoult, R. Schwartzi', and G. Zavaliagkos tBBN Systems and Technologies, Cambridge, MA 02138 ,Northeastern University, Boston, MA 02115 ABSTRACT We present the concept of a "Segmental Neural Net" (SNN) for phonetic modeling in continuous speech recognition. The SNN takes as input all the frames of a phonetic segment and gives as output an estimate of the probability of each of the phonemes, given the input segment. By tak- ing into account all the frames of a phonetic seg- ment simultaneously, the SNN overcomes the well- known conditional-independence limitation of hid- den Markov models (HMM). However, the prob- lem of automatic segmentation with neural nets is a formidable computing task compared to HMMs. Therefore, to take advantage of the training and decoding speed of HMMs, we have developed a novel hybrid SNN/HMM system that combines the advantages of both types of approaches. In this hy- brid system, use is made of the N-best paradigm to generate likely phonetic segmentations, which are then scored by the SNN. The HMM and SNN scores are then combined to optimize performance. In this manner, the recognition accuracy is guaran- teed to be no worse than the HMM system alone. 1 Introduction The current state of the art in continuous speech recognition (CSR) is based on the use of HMMs to model phonemes in context. Two main rea- sons for the popularity of HMMs is their high per- formance, in terms of recognition accuracy, and their computational efficiency (after initial signal processing, real-time recognition is possible on a Sun 4 [1]). However, the limitations of HMMs in modeling the speech signal have been known for some time. Two such limitations are (a) the conditional-independence assumption, which pre- vents a HMM from taking full advantage of the correlation that exists among the frames of a pho- netic segment, and (b) the awkwardness with which segmental features (such as duration) can be incor- porated into HMM systems. We have developed the concept of Segmental Neural Nets (SNN) to overcome the two HMM limitations just mentioned for phonetic modeling in speech. However, neu- ral nets are known to require a large amount of computation, especially for training. Also, there is no known efficient search technique for finding the best scoring segmentation with neural nets in con- tinuous speech. Therefore, we have developed a hybrid SNN/HMM system that is designed to take full advantage of the good properties of both meth- ods: the phonetic modeling properties of SNNs and the good computational properties of HMMs. The two methods are integrated through a novel use of the N-best paradigm developed in conjunction with the BYBLOS system at BBN. 2 Segmental Neural Net Struc- ture There have been several recent approaches to the use of neural nets in CSR. The SNN differs fxom these approaches in that it attempts to recognize each phoneme by using all the frames in a phonetic segment simultaneously to perform the recognition. In fact, we define a …

Content truncated to 500 words