An Empirical Evaluation of Probabilistic Lexicalized Tree Insertion Grammars

An Empirical Evaluation of Probabilistic Lexicalized Tree Insertion Grammars * Rebecca Hwa Harvard University Cambridge, MA 02138 USA rebecca@eecs.harvard.edu Abstract We present an empirical study of the applica- bility of Probabilistic Lexicalized Tree Inser- tion Grammars (PLTIG), a lexicalized counter- part to Probabilistic Context-Free Grammars (PCFG), to problems in stochastic natural- language processing. Comparing the perfor- mance of PLTIGs with non-hierarchical N-gram models and PCFGs, we show that PLTIG com- bines the best aspects of both, with language modeling capability comparable to N-grams, and improved parsing performance over its non- lexicalized counterpart. Furthermore, train- ing of PLTIGs displays faster convergence than PCFGs. 1 Introduction There are many advantages to expressing a grammar in a lexicalized form, where an ob- servable word of the language is encoded in each grammar rule. First, the lexical words help to clarify ambiguities that cannot be re- solved by the sentence structures alone. For example, to correctly attach a prepositional phrase, it is often necessary to consider the lex- ical relationships between the head word of the prepositional phrase and those of the phrases it might modify. Second, lexicalizing the gram- mar rules increases computational efficiency be- cause those rules that do not contain any ob- served words can be pruned away immediately. The Lexicalized Tree Insertion Grammar for- malism (LTIG) has been proposed as a way to lexicalize context-free grammars (Schabes * This material is based upon work supported by the Na- tional Science Foundation under Grant No, IRI 9712068. We thank Yves Schabes and Stuart Shieber for their guidance; Joshua Goodman for his PCFG code; Lillian Lee and the three anonymous reviewers for their com~ ments on the paper. and Waters, 1994). We now apply a prob- abilistic variant of this formalism, Probabilis- tic Tree Insertion Grammars (PLTIGs), to nat- ural language processing problems of stochas- tic parsing and language modeling. This pa- per presents two sets of experiments, compar- ing PLTIGs with non-lexicalized Probabilistic Context-Free Grammars (PCFGs) (Pereira and Schabes, 1992) and non-hierarchical N-gram models that use the right branching bracketing heuristics (period attaches high) as their pars- ing strategy. We show that PLTIGs can be in- duced from partially bracketed data, and that the resulting trained grammars can parse un- seen sentences and estimate the likelihood of their occurrences in the language. The experi- ments are run on two corpora: the Air Travel Information System (ATIS) corpus and a sub- set of the Wall Street Journal TreeBank cor- pus. The results show that the lexicalized na- ture of the formalism helps our induced PLTIGs to converge faster and provide a better language model than PCFGs while maintaining compara- ble parsing qualities. Although N-gram models still slightly out-perform PLTIGs on language modeling, they lack high level structures needed for parsing. Therefore, PLTIGs have combined the best of two worlds: the language modeling capability of N-grams and the parse quality of context-free grammars. The rest of the paper is organized as fol- lows: first, we present an overview of the PLTIG formalism; then …

Content truncated to 500 words