Grouping Words Using Statistical Context

Grouping Words Using Statistical Context Christopher C. Huckle * Department of Psychology, 7 George Square, Edinburgh EH8 9JZ, Scotland, U.K. cch@castle, ed. ac. uk Abstract This paper describes the use of statistical analyses of untagged corpora to detect similarities and differences in the mean- ing of words in text. This work is mo- tivated by psychological as well as by computational issues. The limitations of the method of cluster analysis in as- sessing the success of such analyses are discussed, and ongoing research using an alternative unsupervised neural network approach is described. Introduction There has been considerable recent interest in the use of statistical methods for grouping words in large on-line corpora into categories which capture some of our intuitions about the reference of the words we use and the relationships between them (e.g. Brown et al., 1992; Schiitze, 1993). Although they have received most attention from within computational linguistics, such ap- proaches are also of interest from the point of view of psychology. The huge task of developing con- cepts of word meanings is one that human beings readily achieve; we are all generally aware of the similarities and differences between the meanings of words, despite the fact that in many cases these meanings are not amenable to rigourous defini- tion. Whilst supervision may enable children to learn the meanings of a limited number of com- mon words, it seems extremely unlikely that the greater part of our understanding of word mean- ings is achieved in this way. Experimental evi- dence shows (Harris, 1992) that the occurrence of words in young children's language is strongly in- fluenced by the appearance of those words in the speech they hear around them, and it may be that this process continues indefinitely. Such a process would seem to be particularly important when ac- counting for our understanding of abstract words, such as 'similar' and 'justice', which lack concrete *The author is supported by the Carnegie Trust for the Universities of Scotland referents. Despite our difficulty in being able to provide clear definitions for such words, we have strong intuitions about their usage and can read- ily categorize them on the basis of similarity in meaning. This process of developing concepts for abstract words is one which psychological research has tended to ignore. This situation suggests that the learning of the meanings of many words, and their relation to the meanings of other words, may be achieved in an unsupervised fashion, and that our ability to de- velop a categorization for words may be driven, at least in part, by structure latent in the language being learned. Recent work in computational lin- guistics which makes use of statisticM methods to cluster words into groups which reflect their mean- ing is attractive in this context as it potentially provides a means for developing conceptual struc- ture without supervision, without giving any prior information about the language to the system, and without making a priori distinctions between con- crete and abstract words. Supervision and knowledge …

Content truncated to 500 words