Bootstrapping Coreference Classifiers with Multiple Machine Learning Algorithms

Bootstrapping Coreference Classifiers with Multiple Machine Learning Algorithms Vincent Ng and Claire Cardie Department of Computer Science Cornell University Ithaca, NY 14853-7501 {yung,cardie}@cs.cornell.edu Abstract Successful application of multi-view co- training algorithms relies on the ability to factor the available features into views that are compatible and uncorrelated. This can potentially preclude their use on problems such as coreference resolution that lack an obvious feature split. To bootstrap coref- erence classifiers, we propose and eval- uate a single-view weakly supervised al- gorithm that relies on two different learn- ing algorithms in lieu of the two different views required by co-training. In addition, we investigate a method for ranking un- labeled instances to be fed back into the bootstrapping loop as labeled data, aiming to alleviate the problem of performance deterioration that is commonly observed in the course of bootstrapping. 1 Introduction Co-training (Blum and Mitchell, 1998) is a weakly supervised paradigm that learns a task from a small set of labeled data and a large pool of unlabeled data using separate, but redundant views of the data (i.e. using disjoint feature subsets to represent the data). To ensure provable performance guaran- tees, the co-training algorithm assumes as input a set of views that satisfies two fairly strict condi- tions. First, each view must be sufficient for learn- ing the target concept. Second, the views must be conditionally independent of each other given the class. Empirical results on artificial data sets by Muslea et al. (2002) and Nigam and Ghani (2000) confirm that co-training is sensitive to these assump- tions. Indeed, although the algorithm has been ap- plied successfully to natural language processing (NLP) tasks that have a natural view factorization (e.g. web page classification (Blum and Mitchell, 1998) and named entity classification (Collins and Singer, 1999)), there has been little success, and a number of reported problems, when applying co- training to NLP data sets for which no natural fea- ture split has been found (e.g. anaphora resolution (Mueller et al., 2002)). As a result, researchers have begun to investigate co-training procedures that do not require explicit view factorization. Goldman and Zhou (2000) and Steedman et al. (2003b) use two different learning algorithms in lieu of the multiple views required by standard co-training.1 The intuition is that the two learning algorithms can potentially substitute for the two views: different learners have different rep- resentation and search biases and can complement each other by inducing different hypotheses from the data. Despite their similarities, the principles under- lying the Goldman and Zhou and Steedman et al. co-training algorithms are fundamentally different. In particular, Goldman and Zhou rely on hypothesis testing to select new instances to add to the labeled data. On the other hand, Steedman et al. use two learning algorithms that correspond to coarsely dif- ferent features, thus retaining in spirit the advantages 1Steedman et al. (2003b) bootstrap two parsers that use dif- ferent statistical models via co-training. Hence, the two parsers can effectively be viewed as two different learning algorithms. provided …

Content truncated to 500 words