Discriminative Syntactic Language Modeling for Speech Recognition

Proceedings of the 43rd Annual Meeting of the ACL, pages 507–514, Ann Arbor, June 2005. c⃝2005 Association for Computational Linguistics Discriminative Syntactic Language Modeling for Speech Recognition Michael Collins MIT CSAIL mcollins@csail.mit.edu Brian Roark OGI/OHSU roark@cslu.ogi.edu Murat Saraclar Bogazici University murat.saraclar@boun.edu.tr Abstract We describe a method for discriminative training of a language model that makes use of syntactic features. We follow a reranking approach, where a baseline recogniser is used to produce 1000-best output for each acoustic input, and a sec- ond “reranking” model is then used to choose an utterance from these 1000-best lists. The reranking model makes use of syntactic features together with a parame- ter estimation method that is based on the perceptron algorithm. We describe exper- iments on the Switchboard speech recog- nition task. The syntactic features provide an additional 0.3% reduction in test–set error rate beyond the model of (Roark et al., 2004a; Roark et al., 2004b) (signifi- cant at p < 0.001), which makes use of a discriminatively trained n-gram model, giving a total reduction of 1.2% over the baseline Switchboard system. 1 Introduction The predominant approach within language model- ing for speech recognition has been to use an n- gram language model, within the “source-channel” or “noisy-channel” paradigm. The language model assigns a probability Pl(w) to each string w in the language; the acoustic model assigns a conditional probability Pa(a|w) to each pair (a, w) where a is a sequence of acoustic vectors, and w is a string. For a given acoustic input a, the highest scoring string under the model is w∗ = arg max w (β log Pl(w) + log Pa(a|w)) (1) where β > 0 is some value that reflects the rela- tive importance of the language model; β is typi- cally chosen by optimization on held-out data. In an n-gram language model, a Markov assumption is made, namely that each word depends only on the previous (n − 1) words. The parameters of the language model are usually estimated from a large quantity of text data. See (Chen and Goodman, 1998) for an overview of estimation techniques for n-gram models. This paper describes a method for incorporating syntactic features into the language model, using discriminative parameter estimation techniques. We build on the work in Roark et al. (2004a; 2004b), which was summarized and extended in Roark et al. (2005). These papers used discriminative methods for n-gram language models. Our approach reranks the 1000-best output from the Switchboard recog- nizer of Ljolje et al. (2003).1 Each candidate string w is parsed using the statistical parser of Collins (1999) to give a parse tree T (w). Information from the parse tree is incorporated in the model using a feature-vector approach: we define Φ(a, w) to be a d-dimensional feature vector which in princi- ple could track arbitrary features of the string w together with the acoustic input a. In this paper we restrict Φ(a, w) to only consider the string w and/or the parse tree T (w) for w. For example, …

Content truncated to 500 words