Applying a Grammar-Based Language Model to a Simplified Broadcast-News Transcription Task

Proceedings of ACL-08: HLT, pages 106–113, Columbus, Ohio, USA, June 2008. c⃝2008 Association for Computational Linguistics Applying a Grammar-based Language Model to a Simplified Broadcast-News Transcription Task Tobias Kaufmann Speech Processing Group ETH Z¨urich Z¨urich, Switzerland kaufmann@tik.ee.ethz.ch Beat Pfister Speech Processing Group ETH Z¨urich Z¨urich, Switzerland pfister@tik.ee.ethz.ch Abstract We propose a language model based on a precise, linguistically motivated grammar (a hand-crafted Head-driven Phrase Structure Grammar) and a statistical model estimating the probability of a parse tree. The language model is applied by means of an N-best rescor- ing step, which allows to directly measure the performance gains relative to the baseline sys- tem without rescoring. To demonstrate that our approach is feasible and beneficial for non-trivial broad-domain speech recognition tasks, we applied it to a simplified German broadcast-news transcription task. We report a significant reduction in word error rate com- pared to a state-of-the-art baseline system. 1 Introduction It has repeatedly been pointed out that N-grams model natural language only superficially: an Nth- order Markov chain is a very crude model of the complex dependencies between words in an utter- ance. More accurate statistical models of natural language have mainly been developed in the field of statistical parsing, e.g. Collins (2003), Charniak (2000) and Ratnaparkhi (1999). Other linguistically inspired language models like Chelba and Jelinek (2000) and Roark (2001) have been applied to con- tinuous speech recognition. These models have in common that they explic- itly or implicitly use a context-free grammar induced from a treebank, with the exception of Chelba and Jelinek (2000). The probability of a rule expansion or parser operation is conditioned on various con- textual information and the derivation history. An important reason for the success of these models is the fact that they are lexicalized: the probability dis- tributions are also conditioned on the actual words occuring in the utterance, and not only on their parts of speech. Most statistical parsers achieve a high ro- bustness with respect to out-of-grammar sentences by allowing for arbitrary derivations and rule expan- sions. On the other hand, they are not suited to reli- ably decide on the grammaticality of a given phrase, as they do not accurately model the linguistic con- straints inherent in natural language. We take a completely different position. In the first place, we want our language model to reliably distinguish between grammatical and ungrammati- cal phrases. To this end, we have developed a pre- cise, linguistically motivated grammar. To distin- guish between common and uncommon phrases, we use a statistical model that estimates the probability of a phrase based on the syntactic dependencies es- tablished by the parser. We achieve some degree of robustness by letting the grammar accept arbitrary sequences of words and phrases. To keep the gram- mar restrictive, such sequences are penalized by the statistical model. Accurate hand-crafted grammars have been ap- plied to speech recognition before, e.g. Kiefer et al. (2000) and van Noord et al. (1999). However, they primarily served as a basis for a speech un- derstanding …

Content truncated to 500 words