Effective use of WordNet Semantics via Kernel-Based Learning

Proceedings of the 9th Conference on Computational Natural Language Learning (CoNLL), pages 1–8, Ann Arbor, June 2005. c⃝2005 Association for Computational Linguistics Effective use of WordNet semantics via kernel-based learning Roberto Basili and Marco Cammisa and Alessandro Moschitti Department of Computer Science University of Rome ”Tor Vergata”, Rome, Italy {basili,cammisa,moschitti}@info.uniroma2.it Abstract Research on document similarity has shown that complex representations are not more accurate than the simple bag-of- words. Term clustering, e.g. using latent semantic indexing, word co-occurrences or synonym relations using a word ontol- ogy have been shown not very effective. In particular, when to extend the similar- ity function external prior knowledge is used, e.g. WordNet, the retrieval system decreases its performance. The critical is- sues here are methods and conditions to integrate such knowledge. In this paper we propose kernel func- tions to add prior knowledge to learn- ing algorithms for document classifica- tion. Such kernels use a term similarity measure based on the WordNet hierarchy. The kernel trick is used to implement such space in a balanced and statistically co- herent way. Cross-validation results show the benefit of the approach for the Support Vector Machines when few training data is available. 1 Introduction The large literature on term clustering, term sim- ilarity and weighting schemes shows that docu- ment similarity is a central topic in Information Re- trieval (IR). The research efforts have mostly been directed in enriching the document representation by using clustering (term generalization) or adding compounds (term specifications). These studies are based on the assumption that the similarity between two documents can be expressed as the similarity be- tween pairs of matching terms. Following this idea, term clustering methods based on corpus term dis- tributions or on external prior knowledge (e.g. pro- vided by WordNet) were used to improve the basic term matching. An example of statistical clustering is given in (Bekkerman et al., 2001). A feature selection tech- nique, which clusters similar features/words, called the Information Bottleneck (IB), was applied to Text Categorization (TC). Such cluster based representa- tion outperformed the simple bag-of-words on only one out of the three experimented collections. The effective use of external prior knowledge is even more difficult since no attempt has ever been suc- cessful to improve document retrieval or text classi- fication accuracy, (e.g. see (Smeaton, 1999; Sussna, 1993; Voorhees, 1993; Voorhees, 1994; Moschitti and Basili, 2004)). The main problem of term cluster based represen- tations seems the unclear nature of the relationship between the word and the cluster information lev- els. Even if (semantic) clusters tend to improve the system recall, simple terms are, on a large scale, more accurate (e.g. (Moschitti and Basili, 2004)). To overcome this problem, hybrid spaces containing terms and clusters were experimented (e.g. (Scott and Matwin, 1999)) but the results, again, showed that the mixed statistical distributions of clusters and terms impact either marginally or even negatively on the overall accuracy. In (Voorhees, 1993; Smeaton, 1999), clusters of synonymous terms as defined in WordNet (WN) (Fellbaum, 1998) were used …

Content truncated to 500 words