Automatic Text Categorization by Unsupervised Learning

Automatic Text Categorization by Unsupervised Learning Youngjoong Ko Department of Computer Science, Sogang University 1 Sinsu-dong, Mapo-gu Seoul, 121-742, Korea kyj@nlpzodiac.sogang.ac.kr, Jungyun Seo Department of Computer Science, Sogang University 1 Sinsu-dong, Mapo-gu Seoul, 121-742, Korea seojy@ccs.sogang.ac.kr Abstract The goal of text categorization is to classify documents into a certain number of pre- defined categories. The previous works in this area have used a large number of labeled training documents for supervised learning. One problem is that it is difficult to create the labeled training documents. While it is easy to collect the unlabeled documents, it is not so easy to manually categorize them for creating training documents. In this paper, we propose an unsupervised learning method to overcome these difficulties. The proposed method divides the documents into sentences, and categorizes each sentence using keyword lists of each category and sentence similarity measure. And then, it uses the categorized sentences for training. The proposed method shows a similar degree of performance, compared with the traditional supervised learning methods. Therefore, this method can be used in areas where low-cost text categorization is needed. It also can be used for creating training documents. Introduction With the rapid growth of the internet, the availability of on-line text information has been considerably increased. As a result, text categorization has become one of the key techniques for handling and organizing text data. Automatic text categorization in the previous works is a supervised learning task, defined as assigning category labels (pre-defined) to text documents based on the likelihood suggested by a training set of labeled documents. However, the previous learning algorithms have some problems. One of them is that they require a large, often prohibitive, number of labeled training documents for the accurate learning. Since the application area of automatic text categorization has diversified from newswire articles and web pages to electronic mails and newsgroup postings, it is a difficult task to create training data for each application area (Nigam K. et al., 1998). In this paper, we propose a new automatic text categorization method based on unsupervised learning. Without creating training documents by hand, it automatically creates training sentence sets using keyword lists of each category. And then, it uses them for training and classifies text documents. The proposed method can provide basic data for creating training documents from collected documents, and can be used in an application area to classify text documents in low cost. We use the χ 2 statistic (Yang Y. et al., 1998) as a feature selection method and the naive Bayes classifier (McCallum A. et al., 1998) as a statistical text classifier. The naive Bayes classifier is one of the statistical text classifiers that use word frequencies as features. Other examples include k-nearest-neighbor (Yang Y. et al., 1994), TFIDF/Roccio (Lewis D.D. et al., 1996), support vector machines (Joachims T. et al., 1998) and decision tree (Lewis D.D. et al., 1994). 1 Proposal: A text categorization scheme The proposed system consists of three modules as shown in Figure 1; a module to …

Content truncated to 500 words