Speech Recognition and the Frequency of Recently Used Words: A Modified Markov Model for Natural Language

SPEECH RECOGNITION AND THE FREQUENCY OF RECENTLY USED WORDS A MODIFIED MARKOV MODEL FOR NATURAL LANGUAGE Roland Kuhn School of Computer Science, MeGill University 805 Sherbrooke St. West, Montreal Abstract Speech recognition systems incorporate a language model which, at each stage of the recognition task, assigns a probabil- ity of occurrence to each word in the vocabulary. A class of Mar- kov langnage models identified by Jclinek has achieved consider-. able success in this domain. A modification of the Markov approach, wblch assigns higher probabilities to recently used words, is proposed and tested against a pure Markov model. Parameter calculation and comparison of the two models both involve use of the LOB CorPus of tagged modern English. 1 Introduction Speech recognition systems consist of two components. An acoustic component matches the most recent acoustic input to words in its vocabulary, producing a list of the most plausible word candidates together with a probability for each. The second component, which incorporates a language model, utilizes the string of previously identified words to estimate for each word in the vocabulary the probability that it will occur next. Each word candidate originally selected by the acoustic component is thus associated with two probabilities, the first based on its resemblance to the observed signal and the second based on the linguistic plausibility of that word occurring immediately after the previously recognized words. Multiplication of these two probabilities produces an overall probability for each word candidate. Our work focuses on the language model incorporated in the second component. The language model we use is based on a class of Markov models identified by Jelinek, the "n-gram" and "Mg-gram" models [Jelinek 1985, 1983]. These models, whose parmneters are calculated from a large training text, produce a reasonable non-zero probability for every word in the vocabulary during every stage of the speech recognition task. Our model incorporates both a Markov 3g-gram component and an added "cache" component which tracks short-term fluctuations in word frequency. We adopted the hypothesis that a word used in the recent past is much more likely to be used soon than either its overall frequency in the language or a Markov model would suggest. The cache component of our model estimates the probability of a word from its recent frequency of use. The overall model uses a weighted average of the Markov and cache components in calculating word probabilities, where the relative weights assigned to each component depend on the part of speech (POS). For each POS, the overall model may therefore place more reliance on the cache component than on the Markov component, or vice veins; the relative weights arc obtained empirically for each POS from a training text. This dependance on POS arises from the hypothesis that a content word, such as a particular noun or verb, will occur in bursts. Function wm'ds, on the other hand, would be spread more evenly across a text or a conversation; their short-term frequencies of use would vary less dramatically from their long-term frequencies. …

Content truncated to 500 words