Querying Word Embeddings for Similarity and Relatedness

Proceedings of NAACL-HLT 2018, pages 675–684 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Querying Word Embeddings for Similarity and Relatedness Fatemeh Torabi Asr Simon Fraser University Buranby, BC, Canada ftorabia@sfu.ca Robert Zinkov University of Oxford Oxford, UK zinkov@robots.ox.ac.uk Michael N. Jones Indiana University Bloomington, IN, USA jonesmn@indiana.edu Abstract Word embeddings obtained from neural net- work models such as Word2Vec Skipgram have become popular representations of word meaning and have been evaluated on a vari- ety of word similarity and relatedness norm- ing data. Skipgram generates a set of word and context embeddings, the latter typically discarded after training. We demonstrate the usefulness of context embeddings in predict- ing asymmetric association between words from a recently published dataset of produc- tion norms (Jouravlev and McRae, 2016). Our findings suggest that humans respond with words closer to the cue within the context em- bedding space (rather than the word embed- ding space), when asked to generate themati- cally related words. 1 Introduction Modern distributional semantic models such as Word2Vec (Mikolov et al., 2013a,b) and GloVe (Pennington et al., 2014) have been evaluated on a variety of word similarity and relatedness datasets. A considerable amount of attention has been paid to what models and, more recently, what parameter settings and input data produce embedding representations that better reflect sim- ilarity/relatedness between words, taking human normative judgments as the gold standard (Baroni et al., 2014; Kiela et al., 2015; Levy et al., 2015; Melamud et al., 2016; Sahlgren and Lenci, 2016). Similarity between two words is often assumed to be a direction-less measure (e.g., car and truck are similar due to feature overlap), whereas relat- edness is inherently directional (e.g., broom and floor share a functional relationship). In addition, it is well established in human behavioral data that similarity and relatedness judgments are both asymmetric. For example, humans judge leop- ard to be much more similar to tiger than tiger is to leopard (Tversky and Gati, 1982). A concor- dant asymmetry is seen in relation tasks: in free association data, baby is a much more likely re- sponse when cued with stork than stork would be as a response when cued with baby (Nelson et al., 1999). The distinction between similarity and re- latedness, and the asymmetry of the judgments have typically been ignored in recent evaluations of popular embedding models. There is ample experimental evidence in the psycholinguistic literature that similarity and relat- edness are both well represented in human behav- ior (see Hutchison (2003), for a review), and are qualitatively distinct representations or processes. In semantic priming paradigms, a target word is processed more efficiently when briefly preceded by a related or similar word (e.g., honey-bee or wasp-bee) relative to a neutral or unrelated prime (e.g., chair-bee). Facilitation is seen for word pairs that are purely category coordinates (lawyer- surgeon) or purely associates (scalpel-surgeon), and pairs that share both types of relations (nurse- surgeon) tend to see an additive processing ben- efit that reflects the privilege …

Content truncated to 500 words