Modeling Consensus: Classifier Combination for Word Sense Disambiguation

Modeling Consensus: Classifier Combination for Word Sense Disambiguation Radu Florian and David Yarowsky Department of Computer Science and Center for Language and Speech Processing Johns Hopkins University Baltimore, MD 21218, USA {rflorian,yarowsky}@cs.jhu.edu Abstract This paper demonstrates the substantial empirical success of classifier combination for the word sense disambiguation task. It investigates more than 10 classifier combination methods, including second order classifier stacking, over 6 major structurally different base classifiers (enhanced Naïve Bayes, cosine, Bayes Ratio, decision lists, transformation- based learning and maximum variance boosted mix- ture models). The paper also includes in-depth per- formance analysis sensitive to properties of the fea- ture space and component classifiers. When eval- uated on the standard SENSEVAL1 and 2 data sets on 4 languages (English, Spanish, Basque, and Swedish), classifier combination performance ex- ceeds the best published results on these data sets. 1 Introduction Classifier combination has been extensively stud- ied in the last decade, and has been shown to be successful in improving the performance of diverse NLP applications, including POS tagging (Brill and Wu, 1998; van Halteren et al., 2001), base noun phrase chunking (Sang et al., 2000), parsing (Hen- derson and Brill, 1999) and word sense disambigua- tion (Kilgarriff and Rosenzweig, 2000; Stevenson and Wilks, 2001). There are several reasons why classifier combination is useful. First, by consulting the output of multiple classifiers, the system will im- prove its robustness. Second, it is possible that the problem can be decomposed into orthogonal feature spaces (e.g. linguistic constraints and word occur- rence statistics) and it is often better to train dif- ferent classifiers in each of the feature spaces and then combine their output, instead of designing a complex system that handles the multimodal infor- mation. Third, it has been shown by Perrone and Cooper (1993) that it is possible to reduce the clas- sification error by a factor of � � (� is the number of classifiers) by combination, if the classifiers’ errors are uncorrelated and unbiased. The target task studied here is word sense disam- biguation in the SENSEVAL evaluation framework (Kilgarriff and Palmer, 2000; Edmonds and Cotton, 2001) with comparative tests in English, Spanish, Swedish and Basque lexical-sample sense tagging over a combined sample of 37730 instances of 234 polysemous words. This paper offers a detailed comparative evalu- ation and description of the problem of classifier combination over a structurally and procedurally diverse set of six both well established and orig- inal classifiers: extended Naïve Bayes, BayesRa- tio, Cosine, non-hierarchical Decision Lists, Trans- formation Based Learning (TBL), and the MMVC classifiers, briefly described in Section 4. These systems have different space-searching strategies, ranging from discriminant functions (BayesRatio) to data likelihood (Bayes, Cosine) to decision rules (TBL, Decision Lists), and therefore are amenable 2to combination. Previous Work Related work in classifier combination is discussed throughout this article. For the specific task of word sense disambiguation, the first empirical study was presented in Kilgarriff and Rosenzweig (2000), where the authors combined the output of the par- ticipating SENSEVAL1 systems via simple …

Content truncated to 500 words