Recovering Latent Information in Treebanks

Recovering latent information in treebanks David Chiang and Daniel M. Bikel University of Pennsylvania Dept of Computer and Information Science 200 S 33rd Street Philadelphia PA 19104 USA {dchiang,dbikel}@cis.upenn.edu Abstract Many recent statistical parsers rely on a preprocess- ing step which uses hand-written, corpus-specific rules to augment the training data with extra infor- mation. For example, head-finding rules are used to augment node labels with lexical heads. In this paper, we provide machinery to reduce the amount of human effort needed to adapt existing models to new corpora: first, we propose a flexible notation for specifying these rules that would allow them to be shared by different models; second, we report on an experiment to see whether we can use Expectation- Maximization to automatically fine-tune a set of hand-written rules to a particular corpus. 1 Introduction Most work in statistical parsing does not operate in the realm of parse trees as they appear in many treebanks, but rather on trees transformed via aug- mentation of their node labels, or some other trans- formation (Johnson, 1998). This methodology is il- lustrated in Figure 1. The information included in the node labels’ augmentations may include lexical items, or a node label suffix to indicate the node is an argument and not an adjunct; such extra information may be viewed as latent information, in that it is not directly present in the treebank parse trees, but may be recovered by some means. The process of recov- ering this latent information has largely been limited to the hand-construction of heuristics. However, as is often the case, hand-constructed heuristics may not be optimal or very robust. Also, the effort re- quired to construct such rules can be considerable. In both respects, the use of such rules runs counter to the data-driven approach to statistical parsing. In this paper, we propose two steps to address this problem. First, we define a new, fairly simple syntax for the identification and transformation of node labels that accommodates a wide variety of node-label augmentations, including all those that Model parsed data + annotated data + annotated data Training Decoding parsed data Figure 1: Methodology for the development of a sta- tistical parser. A + indicates augmentation. are performed by existing statistical parsers that we have examined. Second, we explore a novel use of Expectation-Maximization (Dempster et al., 1977) that iteratively reestimates a parsing model using the augmenting heuristics as a starting point. Specif- ically, the EM algorithm we use is a variant of the Inside-Outside algorithm (Baker, 1979; Lari and Young, 1990; Hwa, 1998). The reestimation adjusts the model’s parameters in the augmented parse-tree space to maximize the likelihood of the observed (incomplete) data, in the hopes of finding a better distribution over augmented parse trees (the com- plete data). The ultimate goal of this work is to mini- mize the human effort needed when adapting a pars- ing model to a new domain. 2 Background 2.1 Head-lexicalization Many of the recent, successful statistical parsers have made use of …

Content truncated to 500 words