WAX: A New Dataset for Word Association eXplanations

Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 106–120 November 20–23, 2022. ©2022 Association for Computational Linguistics 106 WAX: A New Dataset for Word Association eXplanations Chunhua Liu† Trevor Cohn† Simon De Deyne‡ Lea Frermann † †School of Computing and Information Systems ‡Melbourne School of Psychological Sciences The University of Melbourne chunhua@student.unimelb.edu.au {tcohn,simon.dedeyne,lfrermann}@unimelb.edu.au Abstract Word associations are among the most com- mon paradigms for studying the human mental lexicon. While their structure and types of as- sociations have been well studied, surprisingly little attention has been given to the question of why participants produce the observed asso- ciations. Answering this question would not only advance understanding of human cogni- tion, but could also aid machines in learning and representing basic commonsense knowl- edge. This paper introduces a large, crowd- sourced dataset of English word associations with explanations, labeled with high-level re- lation types. We present an analysis of the provided explanations, and design several tasks to probe to what extent current pre-trained lan- guage models capture the underlying relations. Our experiments show that models struggle to capture the diversity of human associations, suggesting WAX is a rich benchmark for com- monsense modeling and generation.1 1 Introduction Word associations (Deese, 1966; Kiss et al., 1973) are a prevalent paradigm in cognitive science for probing the human mental lexicon (Nelson et al., 2004; Fitzpatrick, 2006). They reflect spontaneous human associations between concepts. In a typ- ical study, a participant is presented with a cue word (e.g., bagpipe) and asked to spontaneously produce the words that come to mind in response (music, ...). Through large-scale crowd-sourcing studies covering over 12K cues, 3M responses and thousands of participants, a large word association graph (SWOW; Deyne et al. (2019)) has been con- structed, as a resource of basic human conceptual knowledge. This repository of shared associations can serve as a source of commonsense knowledge as shown recently by incorporating SWOW as knowl- 1Data and code are available at https://github. com/ChunhuaLiu596/WAX bagpipe instrument kilt music annoying red Category Exemplar Thematic HasProperty Function “Bagpipe players often wear kilts” “I find the sound of bagpipes annoying” “The bagpipe is a musical instrument” “Bagpipes can be used to play music” Location “The music that comes from bagpipe is nice.” Emotion Evaluation Figure 1: Excerpt of WAX, which consists of associ- ations between cue words (bagpipe) and associations (kilt, red, ...) together with association explanations (speech bubbles) and discrete relation type labels (edge labels). Some associations are supported by distinct relation types and explanations (e.g., bagpipe→music). edge resource into commonsense reasoning mod- els (Liu et al., 2021). However, existing word association data sets like SWOW only provide cue-association pairs, but do not further distinguish between different types of associations. To fill this gap, we constructed a novel data set to recover the underlying reasons by collecting associations together with free-text ex- planations from participants, and distill high-level relation …

Content truncated to 500 words