Example Selection for Bootstrapping Statistical Parsers

Example Selection for Bootstrapping Statistical Parsers Mark Steedman∗, Rebecca Hwa§, Stephen Clark∗, Miles Osborne∗, Anoop Sarkar¶ Julia Hockenmaier∗, Paul Ruhlen† Steven Baker‡, Jeremiah Crim† ∗School of Informatics, University of Edinburgh {steedman,stephenc,julia,osborne}@cogsci.ed.ac.uk §Institute for Advanced Computer Studies, University of Maryland hwa@umiacs.umd.edu ¶School of Computing Science, Simon Fraser University anoop@cs.sfu.ca †Center for Language and Speech Processing, Johns Hopkins University jcrim@jhu.edu,ruhlen@cs.jhu.edu ‡Department of Computer Science, Cornell University sdb22@cornell.edu Abstract This paper investigates bootstrapping for statis- tical parsers to reduce their reliance on manu- ally annotated training data. We consider both a mostly-unsupervised approach, co-training, in which two parsers are iteratively re-trained on each other’s output; and a semi-supervised approach, corrected co-training, in which a human corrects each parser’s output before adding it to the training data. The selection of labeled training examples is an integral part of both frameworks. We propose several selection methods based on the criteria of minimizing er- rors in the data and maximizing training util- ity. We show that incorporating the utility cri- terion into the selection method results in better parsers for both frameworks. 1 Introduction Current state-of-the-art statistical parsers (Collins, 1999; Charniak, 2000) are trained on large annotated corpora such as the Penn Treebank (Marcus et al., 1993). How- ever, the production of such corpora is expensive and labor-intensive. Given this bottleneck, there is consider- able interest in (partially) automating the annotation pro- cess. To overcome this bottleneck, two approaches from ma- chine learning have been applied to training parsers. One is sample selection (Thompson et al., 1999; Hwa, 2000; Tang et al., 2002), a variant of active learning (Cohn et al., 1994), which tries to identify a small set of unlabeled sen- tences with high training utility for the human to label1. Sentences with high training utility are those most likely to improve the parser. The other approach, and the fo- cus of this paper, is co-training (Sarkar, 2001), a mostly- unsupervised algorithm that replaces the human by hav- ing two (or more) parsers label training examples for each other. The goal is for both parsers to improve by boot- strapping off each other’s strengths. Because the parsers may label examples incorrectly, only a subset of their out- put, chosen by some selection mechanism, is used in or- der to minimize errors. The choice of selection method significantly affects the quality of the resulting parsers. We investigate a novel approach of selecting training examples for co-training parsers by incorporating the idea of maximizing training utility from sample selection. The selection mechanism is integral to both sample selection and co-training; however, because co-training and sam- ple selection have different goals, their selection methods focus on different criteria: co-training typically favors se- lecting accurately labeled examples, while sample selec- tion typically favors selecting examples with high train- ing utility, which often are not sentences that the parsers already label accurately. In this work, we investigate se- lection methods for co-training that explore the trade-off between maximizing training utility and minimizing er- rors. Empirical studies were conducted to compare …

Content truncated to 500 words