Method for Improving Automatic Word Categorization

A METHOD FOR IMPROVING AUTOMATIC WORD CATEGORIZATION Emin Erkan Korkmaz GSktfirk U~oluk Department of Computer Engineering Middle East Technical University Emails: korkmaz@ceng.metu.edu.tr ucoluk@ceng.metu.edu.tr Abstract This paper presents a new approach to automatic word categorization which im- proves both the efficiency of the algorithm and the quality of the formed clusters. The unigram and the bigram statistics of a cor- pus of about two million words are used with an efficient distance function to mea- sure the similarities of words, and a greedy algorithm to put the words into clusters. The notions of fuzzy clustering like cluster prototypes, degree of membership are used to form up the clusters. The algorithm is of unsupervised type and the number of clus- ters are determined at run-time. 1 Introduction Statistical natural language processing is a challeng- ing area in the field of computational natural lan- guage learning. Researchers of this field have an approach to language acquisition in which learning is visualised as developing a generative, stochastic model of language and putting this model into prac- tice (de Marcken, 1996). It has been shown practi- cally that the usage of such an approach can yield better performances for acquiring and representing the structure of language. Automatic word categorization is an important field of application in statistical natural language processing where the process is unsupervised and is carried out by working on n-gram statistics to find out the categories of words. Research in this area points out that it is possible to determine the struc- ture of a natural language by examining the regular- ities of the statistics of the language (Finch, 1993). The organization of this paper is as follows. After the related work in the area of word categorization is presented in section 2, a general background of the categorization process is described in 3 section, which is followed by presentation of newly proposed method. In section 4 the results of the experiments are given. We discuss the relevance of the results and conclude in the last section. 2 Related Work There exists previous work in which the unigram and the bigram statistics are used for automatic word clustering. Here it is concluded that the frequency of single words and the frequencies of occurance of word pairs of a large corpus can give the necessary information to build up the word clusters. Finch (Finch and Chater, 1992), makes use of these bigram statistics for the determination of the weight matrix of a neural network. Brown, (Brown et al., 1992) uses the same bigrams and by means of a greedy algorithm forms the hierarchical clusters of words. Genetic algorithms have also been successfuly used for the categorization process. Lanchorst (Lankhorst, 1994) uses genetic algorithms to deter- mine the members of predetermined classes. The drawback of his work is that the number of classes is determined previous to run-time and the genetic algorithm only searches for the membership of those classes. McMahon and Smith (McMahon and Smith, 1996) also use the mutual information of a …

Content truncated to 500 words