A Comparison of Windowless and Window-Based Computational Association Measures as Predictors of Syntagmatic Human Associations

Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 628–637, Singapore, 6-7 August 2009. c⃝2009 ACL and AFNLP A Comparison of Windowless and Window-Based Computational Association Measures as Predictors of Syntagmatic Human Associations Justin Washtell School of Computing University of Leeds washtell@comp.leeds.ac.uk Katja Markert School of Computing University of Leeds markert@comp.leeds.ac.uk Abstract Distance-based (windowless) word asso- cation measures have only very recently appeared in the NLP literature and their performance compared to existing win- dowed or frequency-based measures is largely unknown. We conduct a large- scale empirical comparison of a variety of distance-based and frequency-based mea- sures for the reproduction of syntagmatic human assocation norms. Overall, our results show an improvement in the pre- dictive power of windowless over win- dowed measures. This provides support to some of the previously published the- oretical advantages and makes window- less approaches a promising avenue to explore further. This study also serves as a first comparison of windowed meth- ods across numerous human association datasets. During this comparison we also introduce some novel variations of window-based measures which perform as well as or better in the human association norm task than established measures. 1 Introduction Automatic discovery of semantically associated words has attracted a large amount of attention in the last decades and a host of computational asso- ciation measures have been proposed to deal with this task (see Section 2). These measures tradi- tionally rely on the co-ocurrence frequency of two words in a corpus to estimate a relatedness score. There has been a recent emergence of distance- based language modelling techiques in NLP (Sav- icki and Hlavacova, 2002; Terra and Clarke, 2004) in which the number of tokens separating words is the essential quantity. While some of this work has considered distance-based alternatives to con- ventional association measures (Hardcastle, 2005; Washtell, 2009), there has been no principled em- pirical evaluation of these measures as predictors of human association. We remedy this by conduct- ing a thorough comparison of a wide variety of frequency-based and distance-based measures as predictors of human association scores as elicited in several different free word association tasks. In this work we focus on first-order associ- ation measures as predictors of syntagmatic as- sociations. This is in contrast to second and higher-order measures which are better predictors of paradigmatic associations, or word similarity. The distinction between syntagmatic and paradig- matic relationship types is neither exact nor mu- tually exclusive, and many paradigmatic relation- ships can be observed syntagmatically in the text. Roughly in keeping with (Rapp, 2002), we hereby regard paradigmatic assocations as those based largely on word similarity (i.e. including those typically classed as synonyms, antonyms, hyper- nyms, hyponyms etc), whereas syntagmatic as- sociations are all those words which strongly in- voke one another yet which cannot readily be said to be similar. Typically these will have an identifiable semantic or grammatical relationship (meronym/holonym: stem – flower, verb/object: eat – food etc), or may have harder-to-classify top- ical or idiomatic relationships (family – …

Content truncated to 500 words