Feature2Vec: Distributional semantic modelling of human property knowledge

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 5853–5859, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 5853 Feature2Vec: Distributional semantic modelling of human property knowledge Steven Derby Paul Miller Barry Devereux Queen’s University Belfast, Belfast, United Kingdom {sderby02, p.miller, b.devereux}@qub.ac.uk Abstract Feature norm datasets of human conceptual knowledge, collected in surveys of human volunteers, yield highly interpretable models of word meaning and play an important role in neurolinguistic research on semantic cog- nition. However, these datasets are limited in size due to practical obstacles associated with exhaustively listing properties for a large number of words. In contrast, the develop- ment of distributional modelling techniques and the availability of vast text corpora have allowed researchers to construct effective vec- tor space models of word meaning over large lexicons. However, this comes at the cost of interpretable, human-like information about word meaning. We propose a method for map- ping human property knowledge onto a dis- tributional semantic space, which adapts the word2vec architecture to the task of modelling concept features. Our approach gives a mea- sure of concept and feature affinity in a sin- gle semantic space, which makes for easy and efficient ranking of candidate human-derived semantic properties for arbitrary words. We compare our model with a previous approach, and show that it performs better on several evaluation tasks. Finally, we discuss how our method could be used to develop efficient sampling techniques to extend existing feature norm datasets in a reliable way. 1 Introduction Distributional semantic modelling of word mean- ing has become a popular method for building pretrained lexical representations for downstream Natural Language Processing (NLP) tasks (Baroni and Lenci, 2010; Mikolov et al., 2013; Penning- ton et al., 2014; Levy and Goldberg, 2014; Bo- janowski et al., 2017). In this approach, meaning is encoded in a dense vector space model, such that words (or concepts) that have vector representa- tions that are spatially close together are similar in meaning. A criticism of these real-valued embed- ding vectors is the opaqueness of their representa- tional dimensions and their lack of cognitive plau- sibility and interpretability (Murphy et al., 2012; S¸enel et al., 2018). In contrast, human conceptual property knowledge is often modelled in terms of relatively sparse and interpretable vectors, based on verbalizable, human-elicited features collected in property knowledge surveys (McRae et al., 2005; Devereux et al., 2014). However, gather- ing and collating human-elicited property knowl- edge for concepts is very labour intensive, limiting both the number of words for which a rich fea- ture set can be gathered, as well as the complete- ness of the feature listings for each word. Neural embedding models, on the other hand, learn from large corpora of text in an unsupervised fashion, allowing very detailed, high-dimensional semantic models to be constructed for a very large number of words. In this paper, we propose Feature2Vec, a computational framework that combines infor- mation from human-elicited …

Content truncated to 500 words