Domain Kernels for Text Categorization

Proceedings of the 9th Conference on Computational Natural Language Learning (CoNLL), pages 56–63, Ann Arbor, June 2005. c⃝2005 Association for Computational Linguistics Domain Kernels for Text Categorization Alfio Gliozzo and Carlo Strapparava ITC-Irst via Sommarive, I-38050, Trento, ITALY {gliozzo,strappa}@itc.it Abstract In this paper we propose and evaluate a technique to perform semi-supervised learning for Text Categorization. particular we defined a kernel function,In namely the Domain Kernel, that allowed us to plug “external knowledge” into the supervised learning process. External knowledge is acquired from unlabeled data in a totally unsupervised way, and it is represented by means of Domain Mod- els. We evaluated the Domain Kernel in two standard benchmarks for Text Categoriza- tion with good results, and we compared its performance with a kernel function that exploits a standard bag-of-words feature representation. The learning curves show that the Domain Kernel allows us to re- duce drastically the amount of training data required for learning. 1 Introduction Text Categorization (TC) deals with the problem of assigning a set of category labels to documents. Cat- egories are usually defined according to a variety of topics (e.g. SPORT vs. POLITICS) and a set of hand tagged examples is provided for training. In the state-of-the-art TC settings supervised classifiers are used for learning and texts are represented by means of bag-of-words. Even if, in principle, supervised approaches reach the best performance in many Natural Language Processing (NLP) tasks, in practice it is not always easy to apply them to concrete applicative settings. In fact, supervised systems for TC require to be trained a large amount of hand tagged texts. This situation is usually feasible only when there is some- one (e.g. a big company) that can easily provide al- ready classified documents to train the system. In most of the cases this scenario is quite unprac- tical, if not infeasible. An example is the task of categorizing personal documents, in which the cate- gories can be modified according to the user’s inter- ests: new categories are often introduced and, pos- sibly, the available labeled training for them is very limited. In the NLP literature the problem of providing large amounts of manually annotated data is known as the Knowledge Acquisition Bottleneck. Cur- rent research in supervised approaches to NLP often deals with defining methodologies and algorithms to reduce the amount of human effort required for col- lecting labeled examples. A promising direction to solve this problem is to provide unlabeled data together with labeled texts to help supervision. In the Machine Learning lit- erature this learning schema has been called semi- supervised learning. It has been applied to the TC problem using different techniques: co-training (Blum and Mitchell, 1998), EM-algorithm (Nigam et al., 2000), Transduptive SVM (Joachims, 1999b) and Latent Semantic Indexing (Zelikovitz and Hirsh, 2001). In this paper we propose a novel technique to per- form semi-supervised learning for TC. The under- lying idea behind our approach is that lexical co- 56 herence (i.e. co-occurence in texts of semantically related terms) (Magnini et …

Content truncated to 500 words