Applying Co-Training Methods to Statistical Parsing

Applying Co-Training methods to Statistical Parsing � Anoop Sarkar Dept. of Computer and Information Science University of Pennsylvania 200 South 33rd Street, Philadelphia, PA 19104-6389 USA anoop@linc.cis.upenn.edu Abstract We propose a novel Co-Training method for statistical parsing. The algorithm takes as input a small corpus (9695 sentences) annotated with parse trees, a dictionary of possible lexicalized structures for each word in the training set and a large pool of unlabeled text. The algo- rithm iteratively labels the entire data set with parse trees. Using empirical results based on parsing the Wall Street Journal corpus we show that training a statistical parser on the combined labeled and unlabeled data strongly out- performs training only on the labeled data. 1 Introduction The current crop of statistical parsers share a similar training methodology. They train from the Penn Tree- bank (Marcus et al., 1993); a collection of 40,000 sen- tences that are labeled with corrected parse trees (ap- proximately a million word tokens). In this paper, we explore methods for statistical parsing that can be used to combine small amounts of labeled data with unlimited amounts of unlabeled data. In the experiment reported here, we use 9695 sentences of bracketed data (234467 word tokens). Such methods are attractive for the follow- ing reasons: � Bracketing sentences is an expensive process. A parser that can be trained on a small amount of la- beled data will reduce this annotation cost. � Creating statistical parsers for novel domains and new languages will become easier. � Combining labeled data with unlabeled data allows exploration of unsupervised methods which can now be tested using evaluations compatible with su- pervised statistical parsing. In this paper we introduce a new approach that com- bines unlabeled data with a small amount of labeled (bracketed) data to train a statistical parser. We use a Co- Training method (Yarowsky, 1995; Blum and Mitchell, � I would like to thank Aravind Joshi, Mitch Marcus, Mark Liberman, B. Srinivas, David Chiang and the anonymous reviewers for helpful comments on this work. This work was partially supported by NSF Grant SBR8920230, ARO Grant DAAH0404-94-G-0426, and DARPA Grant N66001-00-1-8915. 1998; Goldman and Zhou, 2000) that has been used pre- viously to train classifiers in applications like word-sense disambiguation (Yarowsky, 1995), document classifica- tion (Blum and Mitchell, 1998) and named-entity recog- nition (Collins and Singer, 1999) and apply this method to the more complex domain of statistical parsing. 2 Unsupervised techniques in language processing While machine learning techniques that exploit anno- tated data have been very successful in attacking prob- lems in NLP, there are still some aspects which are con- sidered to be open issues: � Adapting to new domains: training on one domain, testing (using) on another. � Higher performance when using limited amounts of annotated data. � Separating structural (robust) aspects of the prob- lem from lexical (sparse) ones to improve perfor- mance on unseen data. In the particular domain of statistical parsing there has been limited success in moving towards unsupervised machine learning …

Content truncated to 500 words