Head-Driven Parsing for Word Lattices

Head-Driven Parsing for Word Lattices Christopher Collins Department of Computer Science University of Toronto Toronto, ON, Canada ccollins@cs.utoronto.ca Bob Carpenter Alias I, Inc. Brooklyn, NY, USA carp@alias-i.com Gerald Penn Department of Computer Science University of Toronto Toronto, ON, Canada gpenn@cs.utoronto.ca Abstract We present the first application of the head-driven statistical parsing model of Collins (1999) as a si- multaneous language model and parser for large- vocabulary speech recognition. The model is adapted to an online left to right chart-parser for word lattices, integrating acoustic, n-gram, and parser probabilities. The parser uses structural and lexical dependencies not considered by n- gram models, conditioning recognition on more linguistically-grounded relationships. Experiments on the Wall Street Journal treebank and lattice cor- pora show word error rates competitive with the standard n-gram language model while extracting additional structural information useful for speech understanding. 1 Introduction The question of how to integrate high-level knowl- edge representations of language with automatic speech recognition (ASR) is becoming more impor- tant as (1) speech recognition technology matures, (2) the rate of improvement of recognition accu- racy decreases, and (3) the need for additional in- formation (beyond simple transcriptions) becomes evident. Most of the currently best ASR systems use an n-gram language model of the type pioneered by Bahl et al. (1983). Recently, research has begun to show progress towards application of new and bet- ter models of spoken language (Hall and Johnson, 2003; Roark, 2001; Chelba and Jelinek, 2000). Our goal is integration of head-driven lexical- ized parsing with acoustic and n-gram models for speech recognition, extracting high-level structure from speech, while simultaneously selecting the best path in a word lattice. Parse trees generated by this process will be useful for automated speech understanding, such as in higher semantic parsing (Ng and Zelle, 1997). Collins (1999) presents three lexicalized models which consider long-distance dependencies within a sentence. Grammar productions are conditioned on headwords. The conditioning context is thus more focused than that of a large n-gram covering the same span, so the sparse data problems arising from the sheer size of the parameter space are less press- ing. However, sparse data problems arising from the limited availability of annotated training data be- come a problem. We test the head-driven statistical lattice parser with word lattices from the NIST HUB-1 corpus, which has been used by others in related work (Hall and Johnson, 2003; Roark, 2001; Chelba and Je- linek, 2000). Parse accuracy and word error rates are reported. We present an analysis of the ef- fects of pruning and heuristic search on efficiency and accuracy and note several simplifying assump- tions common to other reported experiments in this area, which present challenges for scaling up to real- world applications. This work shows the importance of careful al- gorithm and data structure design and choice of dynamic programming constraints to the efficiency and accuracy of a head-driven probabilistic parser for speech. We find that the parsing model of Collins (1999) can be successfully adapted as a lan- guage model for …

Content truncated to 500 words