Learning with Unlabeled Data for Text Categorization Using a Bootstrapping and a Feature Projection Technique

Learning with Unlabeled Data for Text Categorization Using Bootstrapping and Feature Projection Techniques Youngjoong Ko Dept. of Computer Science, Sogang Univ. Sinsu-dong 1, Mapo-gu Seoul, 121-742, Korea kyj@nlpzodiac.sogang.ac.kr Jungyun Seo Dept. of Computer Science, Sogang Univ. Sinsu-dong 1, Mapo-gu Seoul, 121-742, Korea seojy@ccs.sogang.ac.kr Abstract A wide range of supervised learning algorithms has been applied to Text Categorization. However, the supervised learning approaches have some problems. One of them is that they require a large, often prohibitive, number of labeled training documents for accurate learning. Generally, acquiring class labels for training data is costly, while gathering a large quantity of unlabeled data is cheap. We here propose a new automatic text categorization method for learning from only unlabeled data using a bootstrapping framework and a feature projection technique. From results of our experiments, our method showed reasonably comparable performance compared with a supervised method. If our method is used in a text categorization task, building text categorization systems will become significantly faster and less expensive. 1 Introduction Text categorization is the task of classifying documents into a certain number of pre-defined categories. Many supervised learning algorithms have been applied to this area. These algorithms today are reasonably successful when provided with enough labeled or annotated training examples. For example, there are Naive Bayes (McCallum and Nigam, 1998), Rocchio (Lewis et al., 1996), Nearest Neighbor (kNN) (Yang et al., 2002), TCFP (Ko and Seo, 2002), and Support Vector Machine (SVM) (Joachims, 1998). However, the supervised learning approach has some difficulties. One key difficulty is that it requires a large, often prohibitive, number of labeled training data for accurate learning. Since a labeling task must be done manually, it is a painfully time-consuming process. Furthermore, since the application area of text categorization has diversified from newswire articles and web pages to E-mails and newsgroup postings, it is also a difficult task to create training data for each application area (Nigam et al., 1998). In this light, we consider learning algorithms that do not require such a large amount of labeled data. While labeled data are difficult to obtain, unlabeled data are readily available and plentiful. Therefore, this paper advocates using bootstrapping framework and a feature projection a technique with just unlabeled data for text categorization. The input to the bootstrapping process is a large amount of unlabeled data and a small amount of seed information to tell the learner about the specific task. In this paper, we consider seed information in the form of title words associated with categories. In general, since unlabeled data are much less expensive and easier to collect than labeled data, our method is useful for text categorization tasks including online data sources such as web pages, E-mails, and newsgroup postings. To automatically build up a text classifier with unlabeled data, we must solve two problems; how we can automatically generate labeled training documents (machine-labeled data) from only title words and how we can handle incorrectly labeled documents in the machine-labeled data. This paper provides solutions for these problems. …

Content truncated to 500 words