When is Self-Training Effective for Parsing?

Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 561–568 Manchester, August 2008 When is Self-Training Effective for Parsing? David McClosky, Eugene Charniak, and Mark Johnson Brown Laboratory for Linguistic Information Processing (BLLIP) Brown University Providence, RI 02912 {dmcc|ec|mj}@cs.brown.edu Abstract Self-training has been shown capable of improving on state-of-the-art parser per- formance (McClosky et al., 2006) despite the conventional wisdom on the matter and several studies to the contrary (Charniak, 1997; Steedman et al., 2003). However, it has remained unclear when and why self- training is helpful. In this paper, we test four hypotheses (namely, presence of a phase transition, impact of search errors, value of non-generative reranker features, and effects of unknown words). From these experiments, we gain a better un- derstanding of why self-training works for parsing. Since improvements from self- training are correlated with unknown bi- grams and biheads but not unknown words, the benefit of self-training appears most in- fluenced by seeing known words in new combinations. 1 Introduction Supervised statistical parsers attempt to capture patterns of syntactic structure from a labeled set of examples for the purpose of annotating new sen- tences with their structure (Bod, 2003; Charniak and Johnson, 2005; Collins and Koo, 2005; Petrov et al., 2006; Titov and Henderson, 2007). These annotations can be used by various higher-level ap- plications such as semantic role labeling (Pradhan et al., 2007) and machine translation (Yamada and Knight, 2001). c⃝ 2008. Licensed under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported li- cense (http://creativecommons.org/licenses/by-nc-sa/3.0/). Some rights reserved. However, labeled training data is expensive to annotate. Given the large amount of unlabeled text available for many domains and languages, tech- niques which allow us to use both labeled and unlabeled text to train our models are desirable. These methods are called semi-supervised. Self- training is a specific type of semi-supervised learn- ing. In self-training, first we train a model on the labeled data and use that model to label the unla- beled data. From the combination of our original labeled data and the newly labeled data, we train a second model – our self-trained model. The pro- cess can be iterated, where the self-trained model is used to label new data in the next iteration. One can think of self-training as a simple case of co- training (Blum and Mitchell, 1998) using a single learner instead of several. Alternatively, one can think of it as one step of the Viterbi EM algorithm. Studies prior to McClosky et al. (2006) failed to show a benefit to parsing from self-training (Char- niak, 1997; Steedman et al., 2003). While the re- cent success of self-training has demonstrated its merit, it remains unclear why self-training helps in some cases but not others. Our goal is to better un- derstand when and why self-training is beneficial. In Section 2, we discuss the previous applica- tions of self-training to parsing. Section 3 de- scribes our experimental setup. We present and test four hypotheses of why self-training helps in Section 4 …

Content truncated to 500 words