A Vector-Based Algorithm for Chinese Text Classification

A Vector-Based Algorithm for Chinese Text ClassificationEll Luo Chang ri Department of Computer Science Central China Normal University Wuhan 430079,PR. China Luocr2@hotmail.com He Ting ting Department of Computer Science Central China Normal University Wuhan 430079,PR. China hett@l63.net Abstract In this paper, vector-distance-weighted algorithm and representative-vector-distance algorithm are described and used to implement the process of automatic text classification. Two experiments have been done by means of the algorithms (experiment) is based on vector-distance-weighted algorithm and experiment2 is based on representative-vector-distance algorithm). Characters are selected as features. The average precision of experiment) and experiment2 is 80.36% and 69.27%, respectively. Comparing the two experiments, it can be concluded that the efficiency of text classification can be improved by means of vector-distance-weighted algorithm. Keywords: text classification, vector-distance-weighted algorithm, Natural Language Process. 1. IntroductionText classification is to assign one or more appropriate class-types for a text based on its content. In western countries, the study about text classification and its relative fields begins as early as 60 or 70s last century, when Salton put forward the VSM (Vector Space Model) theory and the VSM is used successfully in application. The study on Chinese text classification by means of computer starts in the 90s. The research has achieved a lot, for example, FuDan University and Institute of Computing Technology of Chinese Academy of Sciences have tracked and studied the TREC test. Early on PeKing University and TingHua University have made a study of the technology of web classification on their search engine "Network Sky" and "The guide of Network" respectively [LIU Bin, HUANG Tie Jun, CHENG Jun, GAO Wen 2002]. The results of these fields are not as satisfactory as those in English research because of the uniqueness of Chinese language. There are two methods for text classification. One is the rule method. This method is applied earlier. For example, the Construe [Church , K.W.Lisa.F.Rau 1995] is developed based on it. The other method is based on statistics, such as Bayes, VSM, KNN, SVM and so on. In the following section, the authors will discuss the expression of texts in VSM, computation of the feature weight, the similarity between text and classes, and the vector-distance algorithm. At the same time, the results of experiments are figured out and analyzed. 2. Vector Space Model Vector Space Model (VSM) is widely applied in 111. system for its simple conceptions and its simulation of close space to close meanings. The classification method used in texts is introduced from IR system. 2.1. Vector expression of training texts A text is composed of characters, words and phrases which are termed as the features of text. According to "Bayes hypothesis", presuming the effects of features to the class adscription are Ell This paper is supported by Nature Science Foundation of Hubei, China (ID:2001ABBol2) 235 independent, the text can be expressed as the vector of feature collection. After the handling of the training set, "Term-Documents" matrix space Aca could be obtained. The row and column are expressed by the feature and text …

Content truncated to 500 words