Supervised and unsupervised PCFG adaptation to novel domains

Supervised and unsupervised PCFG adaptation to novel domains Brian Roark and Michiel Bacchiani AT&T Labs - Research {roark,michiel}@research.att.com Abstract This paper investigates adapting a lexicalized probabilistic context-free grammar (PCFG) to a novel domain, using maximum a posteriori (MAP) estimation. The MAP framework is gen- eral enough to include some previous model adaptation approaches, such as corpus mixing in Gildea (2001), for example. Other approaches falling within this framework are more effec- tive. In contrast to the results in Gildea (2001), we show F-measure parsing accuracy gains of as much as 2.5% for high accuracy lexicalized pars- ing through the use of out-of-domain treebanks, with the largest gains when the amount of in- domain data is small. MAP adaptation can also be based on either supervised or unsupervised adap- tation data. Even when no in-domain treebank is available, unsupervised techniques provide a sub- stantial accuracy gain over unadapted grammars, as much as nearly 5% F-measure improvement. 1 Introduction A fundamental concern for nearly all data-driven ap- proaches to language processing is the sparsity of la- beled training data. The sparsity of syntactically anno- tated corpora is widely remarked upon, and some recent papers present approaches to improving performance in the absence of large amounts of annotated training data. Johnson and Riezler (2000) looked at adding features to a maximum entropy model for stochastic unification-based grammars (SUBG), from corpora that are not annotated with the SUBG, but rather with simpler treebank annota- tions for which there are much larger treebanks. Hwa (2001) demonstrated how active learning techniques can reduce the amount of annotated data required to converge on the best performance, by selecting from among the candidate strings to be annotated in ways which promote more in- formative examples for earlier annotation. Hwa (1999) and Gildea (2001) looked at adapting parsing models trained on large amounts of annotated data from outside of the domain of interest (out-of-domain), through the use of a relatively small amount of in-domain annotated data. Hwa (1999) used a variant of the inside-outside algorithm presented in Pereira and Schabes (1992) to exploit a partially labeled out-of-domain treebank, and found an advantage to adapta- tion over direct grammar induction. Gildea (2001) simply added the out-of-domain treebank to his in-domain training data, and derived a very small benefit for his high accuracy, lexicalized parser, concluding that even a large amount of out-of-domain data is of little use for lexicalized parsing. Statistical model adaptation based on sparse in-domain data, however, is neither a new problem nor unique to pars- ing. It has been studied extensively by researchers work- ing on acoustic modeling for automatic speech recognition (ASR) (Legetter and Woodland, 1995; Gauvain and Lee, 1994; Gales, 1998; Lamel et al., 2002). One of the meth- ods that has received much attention in the ASR literature is maximum a posteriori (MAP) estimation (Gauvain and Lee, 1994). In MAP estimation, the parameters of the model are considered to be random variables themselves with a known distribution (the prior). The prior distribution and the …

Content truncated to 500 words