Bootstrapping statistical parsers from small datasets

Bootstrapping Statistical Parsers from Small Datasets Mark Steedman*, Miles Osborne*, Anoop Sarkar% Stephen Clark*, Rebecca Hwa. Julia Hockenmaier*, Paul Ruhlent, Steven BakerI and Jeremiah Crimt *Division of Informatics, University of Edinburgh fsteedman,stephenc,julia,osbornel@cogsci.ed.ac.uk 'F School of Computing Science, Simon Fraser University anoop@cs . sfu . ca Institute for Advanced Computer Studies, University of Maryland hwa@umia c s . umd . edu tCenter for Language and Speech Processing, Johns Hopkins University jcrim@jhu.edu ,ruhlen@cs.jhu.edu IDepartment of Computer Science, Cornell University sdb2 2 @cornell .edu Abstract We present a practical co-training method for bootstrapping statistical parsers using a small amount of manu- ally parsed training material and a much larger pool of raw sentences. Experi- mental results show that unlabelled sen- tences can be used to improve the per- formance of statistical parsers. In addi- tion, we consider the problem of boot- strapping parsers when the manually parsed training material is in a differ- ent domain to either the raw sentences or the testing material. We show that boot- strapping continues to be useful, even though no manually produced parses from the target domain are used. 1 Introduction In this paper we describe how co-training (Blum and Mitchell, 1998) can be used to boot- strap a pair of statistical parsers from a small amount of annotated training data. Co-training is a wealdy supervised learning algorithm in which two (or more) learners are iteratively re- trained on each other's output. It has been ap- plied to problems such as word-sense disam- biguation (Yarowsky, 1995), web-page classifica- tion (Blum and Mitchell, 1998) and named-entity recognition (Collins and Singer, 1999). However, these tasks typically involved a small set of la- bels (around 2-3) and a relatively small param- eter space. It is therefore instructive to consider co-training for more complex models. Compared to these earlier models, a statistical parser has a larger parameter space, and instead of class labels, it produces recursively built parse trees as output. Previous work in co-training statistical parsers (Sarkar, 2001) used two components of a single parsing framework (that is, a parser and a supertagger for that parser). In contrast, this paper considers co-training two diverse statistical parsers: the Collins lexicalized PCFG parser and a Lexicalized Tree Adjoining Grammar (LTAG) parser. Section 2 reviews co-training theory. Section 3 considers how co-training applied to training sta- tistical parsers can be made computationally vi- able. In Section 4 we show that co-training out- performs self-training, and that co-training is most beneficial when the seed set of manually created parses is small. Section 4.4 shows that co-training is possible even when the set of initially labelled data is drawn from a different distribution to either the unlabelled training material or the test set; that is, we show that co-training can help in porting a parser from one genre to another. Finally, section 5 reports summary results of our experiments. 2 Co -training: theory Co-training can be informally described in the fol- lowing manner (Blum and Mitchell, 1998): 331 • Pick two (or more) "views" of …

Content truncated to 500 words