A Preliminary Study of Word Clustering Based on Syntactic Behavior

A Preliminary Study of Word Clustering Based on Syntactic Behavior Wide R. Hogenhout and Yuji Matsumoto Nara Institute of Science and Technology 8916-5 Takayama-cho, Ikoma-shi, Nara 630-01, Japan {marc-h ,mat su}~is, aist-nara, ac. jp Abstract We show how a treebank can be used to cluster words on the basis of their syntac- tic behavior. The resulting clusters repre- sent distinct types of behavior with much more precision than parts of speech. As an example we show how prepositions can be automatically subdivided by their syntac- tic behavior and discuss the appropriate- ness of such a subdivision. Applications of this work are also discussed. S works verb NP ~$mith John Smith works fast proper noun proper noun verb adverb Figure 1: Sentence with Parse Tree and Headwords 1 Introduction The construction of classes of words, or calculation of distances between words, has frequently drawn the interest of researchers in natural language pro- cessing. Many of these studies aimed at finding classes based on co-occurrences, often combined with the aim of establishing semantic similarity between words (McMahon and Smith, 1996, Brown et el., 1992, Dagan, Markus, and Markovitch, 1993, Da- gan, Pereira, and Lee, 1994, Pereira and Tishby, 1992, Grefenstette, 1992). We suggest a method for clustering words purely on the basis of syntactic behavior. We show how the necessary data for such clustering can easily be drawn from a publicly available treebank, and how distinct types of behavior can be discovered. Al- though a part of speech tag set can be thought of as a classification based on syntactic behavior, we can construct an arbitrary number of clusters, or a binary tree of words that share their part of speech. We discuss in detail a binary word tree for prepo- sitions that was created by syntactic-behavior based clustering, to show what sort of properties are re- vealed by the clustering and what one can learn from this about language. We also discuss various ways in which this kind of clustering can be used in NLP applications. 2 Headwords and Dependencies The data we extract are based on the concept of headwords. Such headwords are chosen for every constituent in the parse tree by means of a simple set of rules. These have been used in various studies in this field, see (Collins, 1996, Magerman, 1995, Je- linek et el., 1994). Every headword is propagated up through the tree such that every parent receives a headword from the head-child. Figure 1 gives an example of a parse tree with headwords. Following the techniques suggested by (Collins, 1996), a parse tree can subsequently be described as a set of dependencies. Every word except the head- word of the sentence depends on the lowest head- word it is covered by. The syntactic relation is then given by the triple of nonterminals: the modifying nonterminal, the modified nonterminal, and the non- terminal that covers the joint phrase. Table 1 gives an example of such a description. On one point our method is different from the …

Content truncated to 500 words