Find the Funding: Entity Linking with Incomplete Funding Knowledge Bases

Proceedings of the 29th International Conference on Computational Linguistics, pages 178–191 October 12–17, 2022. 178 Exploring Semantic Spaces for Detecting Clustering and Switching in Verbal Fluency Özge Alaçam1, Simeon Schüz1, Martin Wegrzyn2, Johanna Kißler2 and Sina Zarrieß1 1Computational Linguistics, 2Department of Psychology, University of Bielefeld, Bielefeld, Germany {oezge.alacam, simeon.schuez, martin.wegrzyn, johanna.kissler, sina.zarriess}@uni-bielefeld.de Abstract In this work, we explore the fitness of vari- ous word/concept representations in analyzing an experimental verbal fluency dataset provid- ing human responses to 10 different category enumeration tasks. Based on human annota- tions of so-called clusters and switches between sub-categories in the verbal fluency sequences, we analyze whether lexical semantic knowl- edge represented in word embedding spaces (GloVe, fastText, ConceptNet, BERT) is suit- able for detecting these conceptual clusters and switches within and across different categories. Our results indicate that ConceptNet embed- dings, a distributional semantics method en- riched with taxonomical relations, outperforms other semantic representations by a large mar- gin. Moreover, category-specific analysis sug- gests that individual thresholds per category are more suited for the analysis of clustering and switching in particular embedding sub-space instead of a one-fits-all cross-category solution. The results point to interesting directions for fu- ture work on probing word embedding models on the verbal fluency task. 1 Introduction The intrinsic evaluation of lexical knowledge rep- resented in word embeddings has been of long- standing interest in distributional semantics (Levy et al., 2015; Hill et al., 2015), and remains an impor- tant topic in work on interpreting large-scale black- box language models (Pezzelle et al., 2021; Vuli´c et al., 2020; Bommasani et al., 2020). While pre- trained contextualized word representations have recently been evaluated in many novel, (psycho-) linguistically motivated probing tasks (Belinkov and Glass, 2019; Ettinger, 2020; Finlayson et al., 2021), the assessment of lexical semantics in word embeddings still commonly focuses on traditional benchmarks of human similarity annotations (Hill et al., 2015), datasets of analogies (Drozd et al., 2016) or taxonomic relations such as hypernymy (Baroni and Lenci, 2011; Glavaš and Vuli´c, 2018). In this paper, we conduct an evaluation of word embeddings on the so-called verbal fluency task (Shao et al., 2014), where participants are asked to enumerate as many different words for a given category as possible within a given time interval (often 60 seconds), see Figure 1 for an example response to the category hobby. The resulting production data are a rich source of participants’ lexical-conceptual knowledge and typically show an interesting clustering-switching pattern where consecutive words either relate to the same sub- category (handball, football,horseback riding as a cluster for “sports” in Figure 1) or switch between sub-categories (playing the guitar, model railway as a switch from “music” to “playing” in Figure 1). Verbal fluency is a very well-known and widely used cognitive performance test used in psychol- ogy, neuro- and psycholinguistics where robust and automatic methods for analyzing clustering- switching patterns would be highly welcome (Kim et al., 2019). Yet, to date, the analysis of verbal fluency data received little attention in research on computational …

Content truncated to 500 words