Document Clustering Method Based on Frequent Co-occurring Words

Document Clustering Method Based on Frequent Co-occurring Words Ye-Hang Zhu1, Guan-Zhong Dai1, Benjamin C. M. Fung2, De-Jun Mu1, 1 College of Automation, Northwestern Polytechnical University, Xi’an 710072, China 2 School of Computing Science, Simon Fraser University, BC, Canada, V5A 1S6 zhuyehang@yahoo.com.cn Abstract. This paper presents a new document clustering method based on frequent co-occurring words. We first employ the Singular Value Decomposition, and then group the words into clusters called word representatives as substitution of the corresponding words in the original documents. Next, we extract the frequent word representative sets by Apriori. Subsequently, each document is designated to a basic unit described by the frequent word representative set, from which we can get the ultimate clusters by hierarchical clustering. The major advantage of our method is that it can produce the cluster description by the frequent word representatives and then by the corresponding words in the clustering process without any extra works. Compared with the state-of-the-art UPGMA method on benchmark datasets, our method has better performance in terms of the entropy and cluster purity. Keywords: document clustering, text clustering, frequent itemsets, Apriori. 1 Introduction In recent years, the volume of text has growth tremendously due to the Internet, digital libraries, news sources, and company-wide intranets. There is an urgent need to organize them in a structure that facilitates browsing and searching. Clustering [5] is a classic area of machine learning and pattern recognition. There are three major challenges for clustering (hyper)text databases [2]: High dimensionality of the data, large database, and lack of intuitive cluster description. In this paper, we present a novel clustering algorithm based on the idea of frequent words to address these challenges. Fung et al. [4] and Beil et al. [2] introduced a new criterion for hierarchical document clustering using frequent itemsets. In this paper, we use frequent word sets together to illustrate one ultimate cluster. For example, in first cluster the word ‘data’ appear alone and ‘mine’ appear alone also, cannot form frequent 2 word sets, in second cluster the word ‘data’ and ‘mine’ often appear, form frequent 2 word sets, the meaning of the first cluster is much more harder to guess than the second cluster, because when the two word ‘data’ and ‘mine’ constitute phrase ‘data mine’, the meaning of phrase ‘data mine’ is very concrete and definite, so people is easy to associate the second cluster with the concrete meaning: data mine, but the meaning that word ‘data’ alone and word ‘mine’ alone in the first cluster can give is not so definite, so frequent 2 word sets can provide much more information than the word alone that constitute it. 2 Our Clustering Algorithm Below, we describe our clustering algorithm with an example. Step 1 (Preprocessing): Our method employs several preprocessing steps including stop words removal and stemming on the document set. We use SVD(Singular Value Decomposition) on the matrix of word and documents to group similar words to the same cluster [8]. After removing high frequency words and low frequency words, …

Content truncated to 500 words