UNAM at SemEval-2018 Task 10: Unsupervised Semantic Discriminative Attribute Identification in Neural Word Embedding Cones

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 977–984 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics UNAM at SemEval-2018 Task 10: Unsupervised Semantic Discriminative Attribute Identification in Neural Word Embedding Cones Ignacio Arroyo-Fern´andez Universidad Nacional Aut´onoma de M´exico (UNAM) iaf@ccg.unam.mx Ivan Meza Instituto de Investigaciones en Matem´aticas Aplicadas y en Sistemas – UNAM ivanvladimir@turing .iimas.unam.mx Carlos F. M´endez-Cruz Centro de Ciencias Gen´omicas – UNAM cmendezc@ccg.unam.mx Abstract In this paper we report an unsupervised method aimed to identify whether an at- tribute is discriminative for two words (which are treated as concepts, in our par- ticular case). To this end, we use geo- metrically inspired vector operations un- derlying unsupervised decision functions. These decision functions operate on state- of-the-art neural word embeddings of the attribute and the concepts. The main idea can be described as follows: if attribute q discriminates concept a from concept b, then q is excluded from the feature set shared by these two concepts: the inter- section. That is, the membership q ∈ (a ∩ b) does not hold. As a, b, q are represented with neural word embeddings, we tested vector operations allowing us to measure membership, i.e. fuzzy set oper- ations (t-norm, for fuzzy intersection, and t-conorm, for fuzzy union) and the simi- larity between q and the convex cone de- scribed by a and b. 1 Introduction There exist nowadays a number of arithmetic vec- tor operations for computing word relationships interpreted as linguistic regularities. A very pop- ular setting is solving word analogies (Lepage, 1998), which is mainly used to evaluate the quality of word embeddings (Mikolov et al., 2013). Re- cently other alternatives to solve word analogies have been proposed (Linzen, 2016), including su- pervised methods (Drozd et al., 2016). Solving word analogies requires three word ar- guments, and a fourth one is inferred. Such an inference raises from the similarity between com- mon or similar contexts shared by the two pairs of words. Thus, given words “queen”, “woman”, “king”, “man”, the following arithmetic operation holds for their corresponding embeddings x(·): xking − xman + xwoman = xqueen. In this work, we explore similar approaches for Discriminative Attribute Identification (DAI). This task requires tree word arguments a, b, q, and a binary label y ∈ {0, 1} is inferred from them (Cree and McRae, 2003; Lazaridou et al., 2016; McRae et al., 2005). Such a label indi- cates whether the third word, q, is identified as a discriminative (semantic) attribute between words (concepts) a, b. We observed that the task of iden- tifying discriminative attributes between words, represented via word embeddings, evokes that of solving word analogies. We propose geometrically inspired vector oper- ations on word embeddings xa, xb, xq ∈ Rn of the words a, b, q, respectively. The output of each of these operations is in turn operated by a unsuper- vised decision function aimed to predict the label y. The decision functions are based on the reason- ing given originally in (Lepage, …

Content truncated to 500 words