Co-training and Self-training for Word Sense Disambiguation

Co-training and Self-training for Word Sense Disambiguation Rada Mihalcea Department of Computer Science and Engineering University of North Texas rada@cs.unt.edu Abstract This paper investigates the application of co- training and self-training to word sense disam- biguation. Optimal and empirical parameter se- lection methods for co-training and self-training are investigated, with various degrees of error reduction. A new method that combines co- training with majority voting is introduced, with the effect of smoothing the bootstrapping learn- ing curves, and improving the average perfor- mance. 1 Introduction The task of word sense disambiguation consists in assign- ing the most appropriate meaning to a polysemous word within a given context. Most of the efforts in solving this problem were concentrated so far towards supervised learning, where each sense tagged occurrence of a par- ticular word is transformed into a feature vector, which is then used in an automatic learning process. While these algorithms usually achieve the best performance, as compared to their unsupervised or knowledge-based al- ternatives, there is an important shortcoming associated with these methods: their applicability is limited only to those words for which sense tagged data is available, and their accuracy is strongly connected to the amount of la- beled data available at hand. In this paper, we investigate methods for building sense classifiers when only relatively few annotated examples are available. We explore boot- strapping methods using co-training and self-training, and evaluate their performance on the SENSEVAL-2 nouns. We show that classifiers built for different words have dif- ferent behavior during the bootstrapping process. If the right parameters for co-training and self-training can be identified (growth size, pool size, and number of itera- tions, as explained later in the paper), an average error reduction of 25.5% is achieved, with similar performance observed for both co-training and self-training. However, with empirical settings, the error reduction is significantly smaller, with a 9.8% error rate reduction achieved for a new method that combines co-training with majority vot- ing. We first overview the general approach of bootstrap- ping for natural language learning using co-training and self-training. We then introduce the problem of super- vised word sense disambiguation, and define several local and topical basic classifiers. We investigate the applica- bility of co-training and self-training to supervised word sense disambiguation, starting with these basic classifiers, and perform comparative evaluations of optimal and em- pirical bootstrapping parameter settings. 2 Co-training and Self-training for Natural Language Learning Co-training and self-training are bootstrapping methods that aim to improve the performance of a supervised learn- ing algorithm by incorporatinglarge amounts of unlabeled data into the training data set. 2.1 Co-training Starting with a set of labeled data, co-training algorithms (Blum and Mitchell, 1998) attempt to increase the amount of annotated data using some (large) amounts of unlabeled data. Shortly, co-training algorithms work by generating several classifiers trained on the input labeled data, which are then used to tag new unlabeled data. From this newly annotated data, the most confident predictions are sought, and subsequently added to the set …

Content truncated to 500 words