An Empirical Approach to Text Categorization Based on Term Weight Learning

An Empirical Approach to Text Categorization based on Term Weight Learning Fumiyo Fukumoto and Yoshimi Suzukit Department of Computer Science and Media Engineering, Yamanashi University 4-3-11 Takeda, Kofu 400-8511 Japan { fukumoto@skyc,ysuzuki©suwa t} .esi.yamanashi.a.c.jp Abstract In this paper) we propose a method for text categoriza- Lion task using term weight learning. In our approach, learning is to learn true keywords from the error of clus- tering results. Parameters of term weighting are then estimated so as to maximize the true keywords and min- imize the other words in the text. The characteristic of our approach is that the degree of context dependency is used in order to judge whether a word in a text is a true keyvv·ord or not. The experiments using Wall Street Journal corpus demonstrate the effectiveness of the method. Introduction \~Vith increasing numbers of machine readable docti- ments becoming availa.ble, an automatic text catego- rization which is the classification of text with respect t.o a. set. of pre-categori;.:ed texts) has become a trend in IR and NLP studies. .I One of the important issues in text cHtegoriza.- t.ion task is hc)\\1 to characteri;.:e texts whicl.1 are pre- categorized. There are at least two stat.lstica.l ap- proaches t.o cope with the issue) i.e. statistical approach that relies mainly on ( 1) surface information of words in texts, and (2.) senwntic infonnation of \VOrds in texts. Statistical approach based on surface information of words has been widely studied in IR. One represen- tative is a vector model. In this model, each text. is represented by a vector, i.e. every text which should be classified a.nd texts which are pre-categorized in a training phase are characterized by a vector, each di- mension of which is associated with a specific word in Lcxts) and every coordinate of the text is represented by term weighting. Then) some similarity measure is used and the text is assigned to the most sema.n- tica.lly similar set of texts which are pre-ca.t.egori;.:ed. Term weighting method is widely studied [Luhn 1958], [Salton and Yang1973], [Salton\988], [.Jones1973]. Guthrie a.nd Yuasa. used word frequencies for weight- ing [Guthrie and Walkerl994], [Yuasa et al.l995], and Tokunaga used weighted inverse document frequency (WIDF) which is a word frequency within the docu- ment divided by its frequency throughout the entire 71 document collection [Tokunaga and Iwayama1994]. The other approach is based on a probabilistic model. This approach is widely used, since it has solid formal grounding in probability theory. I way am a et. al. proposed a probabilistic rnodel called Sin- gle mndom Variable with Multiple Values (SVMV) [Iwayama and Tokunaga\994]. They reported that t.he result of their experiment using S VM V was better than other probabilistic models; Component Theory(CT) [Kwok\989], Probabilistic Relevance Weighting(PRW) [Robertson and Jones\976] and Retrieval with Proba- bilistic Inde,;ing(RI'I) [Fuhr!989] in the task of catego- rizing news articles from the Wall Street. Journal( W8J). Most previous approaches seem to show the effect in en- tirely difl'erent texts, such as 'weather forecasts', 'medi- cal reports) and 'computer …

Content truncated to 500 words