The Computation of Word Associations: Comparing Syntagmatic and Paradigmatic Approaches

The Computation of Word Associations: Comparing Syntagmatic and Paradigmatic Approaches Reinhard Rapp University of Mainz, FASK D-76711 Germersheim, Germany rapp@mail.fask.uni-mainz.de Abstract It is shown that basic language processes such as the production of free word associations and the generation of synonyms can be simulated using statistical models that analyze the distri- bution of words in large text corpora. Accord- ing to the law of association by contiguity, the acquisition of word associations can be ex- plained by Hebbian learning. The free word as- sociations as produced by subjects on presenta- tion of single stimulus words can thus be pre- dicted by applying first-order statistics to the frequencies of word co-occurrences as observed in texts. The generation of synonyms can also be conducted on co-occurrence data but re- quires second-order statistics. The reason is that synonyms rarely occur together but appear in similar lexical neighborhoods. Both approaches are systematically compared and are validated on empirical data. It turns out that for both tasks the performance of the statistical system is comparable to the performance of human sub- jects. 1 Introduction According to Ferdinand de Saussure (1916), there are two fundamental types of relations between words that he believes correspond to basic opera- tions of our brain: syntagmatic and paradigmatic associations. There is a syntagmatic relation be- tween two words if they co-occur in spoken or written language more frequently than expected from chance and if they have different grammatical roles in the sentences in which they occur. Typical examples are the word pairs coffee – drink, sun – hot, or teacher – school. The relation between two words is paradigmatic if the two words can sub- stitute for one another in a sentence without affect- ing the grammaticality or acceptability of the sen- tence. Typical examples are synonyms or antonyms like quick – fast, or eat – drink. Normally, words with a paradigmatic relation are the same part of speech, whereas words with a syntagmatic relation can but need not be the same part of speech. In this paper we want to show that the two types of relations as defined by de Saussure are reflected in the statistical distribution of words in large cor- pora. We present algorithms that automatically retrieve words with either the syntagmatic or the paradigmatic type of relationship from corpora and perform a quantitative evaluation of our results. 2 Paradigmatic Associations Paradigmatic associations are words with high se- mantic similarity. According to Ruge (1992), the semantic similarity of two words can be computed by determining the agreement of their lexical neighborhoods. For example, the semantic similarity of the words red and blue can be derived from the fact that they both frequently co-occur with words like color, flower, dress, car, dark, bright, beauti- ful, and so forth. If for each word in a corpus a co- occurrence vector is determined whose entries are the co-occurrences with all other words in the cor- pus, then the semantic similarities between words can be computed by conducting simple vector …

Content truncated to 500 words