Speech Recognition Using a Stochastic Language Model Integrating Local and Global Constraints

SPEECH RECOGNITION USING A STOCHASTIC LANGUAGE MODEL INTEGRATING LOCAL AND GLOBAL CONSTRAINTS Ryosuke Isotani, Shoichi Matsunaga ATR Interpreting Telecommunications Research Laboratories Seika-cho, Soraku-gun, Kyoto 619-02, Japan ABSTRACT In this paper, we propose a new stochastic language model that integrates local and global constraints effectively and describe a speechrecognition system basedon it. Theproposedlanguagemodel uses the dependencies within adjacent words as local constraints in the same way as conventional word N-gram models. To capture the global constraints between non-contiguous words, we take into account the sequence of the function words and that of the content words which are expected to represent, respectively, the syntactic and semantic relationships between words. Furthermore, we show that assuming an independence between local- and global constraints, the number of parameters to be estimated and stored is greatly reduced. The proposed language model is incorporated into a speech recog- nizer based on the time-synchronous Viterbi decoding algorithm, and compared with the word bigram model and trigram model. The pro- posed model gives a better recognition rate than the bigram model, though slightly worse than the trigram model, with only twice as many parameters as the bigram model. 1. INTRODUCTION At present, word N-gram models [1], especially bigram (N = 2).or trigram (N = 3) models, are recognized as effective and are widely used as language models for speech recognition. Such models, how- ever, represent only local constraints within a few successive words and lack the ability to capture global or long distance dependencies between words. They might represent global constraints if N were set at a larger value, but it is not only computationally impractical but also inefficient because dependencies between non-contiguous words are often independent of the contents and length of the word string between them. In addition, estimating so many parameters from a finite number of text corpora would result in sparseness of data. Recently some papers treat long distance factors. In the long dis- tance bigrams by Huang et al. [2] a linear combination of distance-d bigrams is used. All the preceding words in a window of fixed length are considered, and bigram probabilities are estimated for each distance d between words respectively. The extended bigram model by Wright et al. [3] uses a single word selected for each word according to a statistical measure as its "parent." The extended bigrams are insensitive to the distance between the word and its parent, but this model does not utilize multiple information. The trigger pairs described in [4, 5] also represent relationships between non-contiguous words. They are also extracted automatically and insensitive to the distance. The way of combining the evidence from trigger pairs with local constraints ("the static model" in their term) is also given. But this approach has the disadvantage that it is com- putationaUy expensive. Another approach is a tree-based model [6], which automatically generates a binary decision tree from training data. Although it could also extract similar dependencies by setting binary questions appropriately, it has the same disadvantage as the trigger-based model. We …

Content truncated to 500 words