Weakly Supervised Natural Language Learning Without Redundant Views

Weakly Supervised Natural Language Learning Without Redundant Views Vincent Ng and Claire Cardie Department of Computer Science Cornell University Ithaca, NY 14853-7501 � yung,cardie � @cs.cornell.edu Abstract We investigate single-view algorithms as an al- ternative to multi-view algorithms for weakly supervised learning for natural language pro- cessing tasks without a natural feature split. In particular, we apply co-training, self-training, and EM to one such task and find that both self- training and FS-EM, a new variation of EM that incorporates feature selection, outperform co- training and are comparatively less sensitive to parameter changes. 1 Introduction Multi-view weakly supervised learning paradigms such as co-training (Blum and Mitchell, 1998) and co-EM (Nigam and Ghani, 2000) learn a classification task from a small set of labeled data and a large pool of unla- beled data using separate, but redundant, views of the data (i.e. using disjoint feature subsets to represent the data). Multi-view learning has been successfully ap- plied to a number of tasks in natural language processing (NLP), including text classification (Blum and Mitchell, 1998; Nigam and Ghani, 2000), named entity classifica- tion (Collins and Singer, 1999), base noun phrase brack- eting (Pierce and Cardie, 2001), and statistical parsing (Sarkar, 2001; Steedman et al., 2003). The theoretical performance guarantees of multi-view weakly supervised algorithms come with two fairly strong assumptions on the views. First, each view must be sufficient to learn the given concept. Second, the views must be conditionally independent of each other given the class label. When both conditions are met, Blum and Mitchell prove that an initial weak learner can be boosted using unlabeled data. Unfortunately, finding a set of views that satisfies both of these conditions is by no means an easy problem. In addition, recent empirical results by Muslea et al. (2002) and Nigam and Ghani (2000) have shown that multi-view algorithms are quite sensitive to the two underlying as- sumptions on the views. Effective view factorization in multi-view learning paradigms, therefore, remains an im- portant issue for their successful application. In practice, views are supplied by users or domain experts, who deter- mine a natural feature split that is expected to be redun- dant (i.e. each view is expected to be sufficient to learn the target concept) and conditionally independent given the class label.1 We investigate here the application of weakly super- vised learning algorithms to problems for which no obvi- ous natural feature split exists and hypothesize that, in these cases, single-view weakly supervised algorithms will perform better than their multi-view counterparts. Motivated, in part, by the results in Mueller et al. (2002), we use the task of noun phrase coreference resolution for illustration throughout the paper.2 In our experi- ments, we compare the performance of the Blum and Mitchell co-training algorithm with that of two com- monly used single-view algorithms, namely, self-training and Expectation-Maximization (EM). In comparison to co-training, self-training achieves substantially superior performance and is less sensitive to its input parameters. EM, on the other hand, fails to boost performance, and we attribute …

Content truncated to 500 words