Manipulating Large Corpora for Text Classification

Manipulating Large Corpora for Text Classification Fumiyo Fukumoto and Yoshimi Suzuki Department of Computer Science and Media Engineering, Yamanashi University 4-3-11 Takeda, Kofu 400-8511 Japan fukumoto@skye.esb.yamanashi.ac.jp ysuzuki@alps1.esi.yamanashi.ac.jp Abstract In this paper, we address the problem of dealing with a large collection of data and propose a method for text classifi- cation which manipulates data using two well-known machine learning techniques, Naive Bayes(NB) and Support Vector Ma- chines(SVMs). NB is based on the as- sumption of word independence in a text, which makes the computation of it far more efficient. SVMs, on the other hand, have the potential to handle large feature spaces, which makes it possible to pro- duce better performance. The training data for SVMs are extracted using NB classifiers according to the category hier- archies, which makes it possible to reduce the amount of computation necessary for classification without sacrificing accuracy. 1 Introduction As the volume of online documents has drastically increased, text classification has become more im- portant, and a growing number of statistical and ma- chine learning techniques have been applied to the task(Lewis, 1992), (Yang and Wilbur, 1995), (Baker and McCallum, 1998), (Lam and Ho, 1998), (Mc- Callum, 1999), (Dumais and Chen, 2000). Most of them use the Reuters-21578 articles1 in the evalu- 1The Reuters-21578, distribution 1.0, is comprised of 21,578 documents, representing what remains of the original Reuters-22173 corpus after the elimination of 595 duplicates by Steve Lynch and David Lewis in 1996. ations of their methods, since the corpus has be- come a benchmark, and their results are thus eas- ily compared with other results. It is generally agreed that these methods using statistical and ma- chine learning techniques are effective for classifi- cation task, since most of them showed significant improvement (the performance over 0.85 F1 score) for Reuters-21578(Joachims, 1998), (Dumais et al., 1998), (Yang and Liu, 1999). More recently, some researchers have applied their techniques to larger corpora such as web pages in Internet applications(Mladenic and Grobel- nik, 1998), (McCallum, 1999), (Dumais and Chen, 2000). The increasing number of documents and categories, however, often hampers the develop- ment of practical classification systems, mainly due to statistical, computational, and representational problems(Dietterich, 2000). There are at least two strategies for solving these problems. One is to use category hierarchies. The idea behind this is that when humans organize extensive data sets into fine-grained categories, category hierarchies are of- ten employed to make the large collection of cate- gories more manageable. McCallum et. al. pre- sented a method called ‘shrinkage’ to improve pa- rameter estimates by taking advantage of a hierar- chy(McCallum, 1999). They tested their method us- ing three different real-world datasets: 20,000 ar- ticles from UseNet, 6,440 web pages from the in- dustry sector, and 14,831 pages from Yahoo, and showed improved performance. Dumais et. al. used SVMs and classified hierarchical web content consisting of 50,078 web pages for training, and 10,024 for testing, with promising results(Dumais Association for Computational Linguistics. Language Processing (EMNLP), Philadelphia, July 2002, pp. 196-203. Proceedings of the …

Content truncated to 500 words