Ensemble Methods for Automatic Thesaurus Extraction

Ensemble Methods for Automatic Thesaurus Extraction James R. Curran Institute for Communicating and Collaborative Systems University of Edinburgh 2 Buccleuch Place, Edinburgh EH8 9LW United Kingdom jamesc@cogsci.ed.ac.uk Abstract Ensemble methods are state of the art for many NLP tasks. Recent work by Banko and Brill (2001) suggests that this would not necessarily be true if very large training corpora were available. However, their results are limited by the simplic- ity of their evaluation task and individual classifiers. Our work explores ensemble efficacy for the more complex task of automatic the- saurus extraction on up to 300 million words. We examine our conflicting results in terms of the constraints on, and com- plexity of, different contextual representa- tions, which contribute to the sparseness- and noise-induced bias behaviour of NLP systems on very large corpora. 1 Introduction Ensemble learning is a machine learning technique that combines the output of several different classi- fiers with the goal of improving classification per- formance. The classifiers within the ensemble may differ in several ways, such as the learning algorithm or knowledge representation used, or data they were trained on. Ensemble learning has been successfully applied to numerous NLP tasks, including POS tag- ging (Brill and Wu, 1998; van Halteren et al., 1998), chunking (Tjong Kim Sang, 2000), word sense dis- ambiguation (Pederson, 2000) and statistical pars- ing (Henderson and Brill, 1999). Dietterich (2000) presents a good introduction to ensemble methods. Ensemble methods ameliorate learner bias by amortising individual classifier bias of over differ- ent systems. For an ensemble to be more effec- tive than its constituents, the individual classifiers must have better than 50% accuracy and must pro- duce diverse erroneous classifications (Dietterich, 2000). Brill and Wu (1998) call this complementary disagreement complementarity. Although ensem- bles are often effective on problems with small train- ing sets, recent work suggests this may not be true as dataset size increases. Banko and Brill (2001) found that for confusion set disambiguation with corpora larger than 100 million words, the best individual classifiers outperformed ensemble methods. One limitation of their results is the simplicity of the task and methods used to examine the efficacy of ensemble methods. However, both the task and applied methods are constrained by the ambitious use of one billion words of training material. Dis- ambiguation is relatively simple because confusion sets are rarely larger than four elements. The indi- vidual methods must be inexpensive because of the computational burden of the massive training set, so they must perform limited processing of the training corpus and can only consider a fairly narrow context surrounding each instance. We explore the value of ensemble methods for the more complex task of automatic thesaurus extrac- tion, training on corpora of up to 300 million words. The increased complexity leads to results contradict- ing Banko and Brill (2001), which we explore using ensembles of different contextual complexity. This work emphasises the link between contextual com- plexity and the problems of representation sparse- ness and noise as corpus size increases, which …

Content truncated to 500 words