Exploring Vector Spaces for Semantic Relations

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1814–1823 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Exploring Vector Spaces for Semantic Relations Kata Gábor1, Haïfa Zargayouna1, Isabelle Tellier2, Davide Buscaldi1, Thierry Charnois1 1 LIPN, CNRS (UMR 7030), Université Paris 13 2 LaTTiCe, CNRS (UMR 8094), ENS Paris, Université Sorbonne Nouvelle, PSL Research University, SPC Abstract Word embeddings are used with success for a variety of tasks involving lexical semantic similarities between individual words. Us- ing unsupervised methods and just cosine similarity, encouraging results were ob- tained for analogical similarities. In this pa- per, we explore the potential of pre-trained word embeddings to identify generic types of semantic relations in an unsupervised experiment. We propose a new relational similarity measure based on the combina- tion of word2vec’s CBOW input and output vectors which outperforms alternative vec- tor representations, when used for unsuper- vised clustering on SemEval 2010 Relation Classification data. 1 Introduction Vector space word representations or word embed- dings, both ’count’ models (Turney and Pantel, 2010) and learned vectors (Mikolov et al., 2013a; Pennington et al., 2014), were proven useful for a variety of semantic tasks (Mikolov et al., 2013b; Baroni et al., 2014). Word vectors are used with success because they capture a notion of seman- tics directly extracted from corpora. Distributional representations allow to compute a functional or topical semantic similarity between two words or, more recently, bigger text units (Le and Mikolov, 2014). The more similar two entities are semanti- cally, the closer they are in the vector space (quan- tified usually, but not necessarily in terms of cosine similarity). Semantic similarity can be exploited for lexical substitution, synonym detection, sub- categorization learning etc. Recent studies sug- gest that neural word embeddings show higher per- formance than count models (Baroni et al., 2014; Krebs and Paperno, 2016) for most semantic tasks, although Levy et al. (2015a) argue that this is only due to some specific hyperparameters that can be adapted to count vectors. In what follows, we will concentrate on exploring whether and how pre- trained, general-purpose word embeddings encode relational similarities. 1.1 Relational analogies as vector offsets Relation extraction and classification deal with identifying the semantic relation linking two en- tities or concepts based on different kinds of in- formation, such as their respective contexts, their co-occurrences in a corpus and their position in an ontology or other kind of semantic hierarchy. Whether the vector spaces of pre-trained word em- beddings are appropriate for discovering or iden- tifying relational similarities remains to be seen. Mikolov et al. (2013b) claimed that the embed- dings created by a recursive neural network indeed encode a specific kind of relational similarities, i.e. analogies between pairs of words. He found that by using simple vector arithmetic, analogy questions in the form of "a1 is to a2 as b1 is to b2" (man ~king :: woman ~queen) could be solved. Relationships are assumed to be present as vector offsets, so that in the embedding …

Content truncated to 500 words