A Probabilistic Model for Text Categorization: Based on a Single Random Variable with Multiple Values

A Probabilistic Model for Text Categorization: Based on a Single Random Variable with Multiple Values Makoto IWAYAMA Advanced Research Laboratory Hitachi Ltd. HATOYAMA SAITAMA 350-03, JAPAN iwayama©harl, hit achi. co. j p Abstract Text categorization is the classification of documents with respect to a set of prede- fined categories. In this paper, we propose a new probabilistic model for text catego- rization, that is based on a Single random Variable with Multiple Values (SVMV). Com- pared to previous probabilistic models, our model has the following advantages; 1) it considers within-document term frequencies, 2) considers term weighting for target docu- ments, and 3) is less affected by having insuf- ficient training cases. We verify our model's superiority over the others in the task of cat- egorizing news articles from the "Wall Street Journal". 1 Introduction Text categorization is the classification of documents with respect to a set of predefined categories. As an example, let us take a look at the following article from the "Wall Street Journal" (1989/11/2). McDermott International, Inc. said its Bab- cock & Wilcox unit completed the sale of its Bailey Controls Operations to Finmeccanica S.p.A for $295 million. Finmeccanica is an Italian state-owned holding company with in- terests in the mechanical engineering indus- try. Bailey Controls, based in Wickliffe, Ohio, makes computerized industrial controls sys- tems. It employs 2,700 people and has annual revenue of about $370 million. Two categories (topics) are manually assigned to this article; "TENDER OFFERS, MERGERS, ACQUISI- TIONS (TNM)" and "COMPUTERS AND INFORMA- TION TECHNOLOGY (CPK)." While there may be cer- tain rules or standards for categorization, it is very difficult for human experts to assign categories con- sistently and efficiently to large numbers of daily in- coming documents. The purpose of this paper is to propose a new probabilistic model for automatic text categorization. Takenobu TOKUNAGA Department of Computer Science Tokyo Institute of Technology 2-12-1, OOKAYAMA, MEGURO-KU TOKYO 152, JAPAN t ake@cs, tit ech. ac. jp While many text categorization models have been proposed so far, in this paper, we concentrate on the probabilistic models (Robertson and Sparck Jones, 1976; Kwok, 1990; Fuhr, 1989; Lewis, 1992; Croft, 1981; Wong and Yao, 1989; Yu et al., 1989) because these models have solid formal grounding in probabil- ity theory. Section 2 quickly reviews the probabilis- tic models and lists their individual problems. In sec- tion 3, we propose a new probabilistic model based on a Single random Variable with Multiple Values (SVMV). Our model is very simple, but solves some problems of the previous models. In section 4, we verify our model's superiority over the others through experi- ments in which we categorize "Wall Street Journal" articles. 2 A Brief Survey of Probabilistic Text Categorization In this section, we will briefly review three major prob- abilistic models for text categorization. Originally, these models have been exploited for information re- trieval, but the adaptation to text categorization is straightforward. In a model of probabilistic text categorization, P(cld ) = "the probability that a document d …

Content truncated to 500 words