HHU at SemEval-2017 Task 2: Fast Hash-Based Embeddings for Semantic Word Similarity Assessment

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 250–255, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics HHU at SemEval-2017 Task 2: Fast Hash-Based Embeddings for Semantic Word Similarity Assessment Behrang Qasemizadeh DFG SFB 991 Universität Düsseldorf Düsseldorf, Germany zadeh@phil.hhu.de Laura Kallmeyer DFG SFB 991 Universität Düsseldorf Düsseldorf, Germany kallmeyer@phil.hhu.de Abstract This paper describes the HHU system that participated in Task 2 of SemEval 2017, Multilingual and Cross-lingual Semantic Word Similarity. We introduce our un- supervised embedding learning technique and describe how it was employed and configured to address the problems of monolingual and multilingual word sim- ilarity measurement. This paper reports from empirical evaluations on the bench- mark provided by the task’s organizers. 1 Introduction The goal of Task 2 of SemEval-2017 is to provide a reliable benchmark for the evaluation of mono- lingual and multilingual semantic representations (Camacho-Collados et al., 2017). The proposed evaluation benchmark goes beyond classic seman- tic relatedness tests by providing both monolin- gual and cross-lingual data sets that include mul- tiword expressions, domain-specific terms, and named entities for five languages. To measure ‘se- mantic similarity’ between pairs of lexical items, the HHU system uses the algorithm proposed in (QasemiZadeh et al., 2017), which is based on a derandomization of the ‘random positive-only projections’ method proposed by QasemiZadeh and Kallmeyer (2016). Word embedding techniques (i.e., using dis- tributional frequencies to produce word vectors of reduced dimensionality) are one of the most popular approaches to semantic word similar- ity problems. These methods are often ratio- nalized using Harris’ Distributional Hypothesis that words of similar linguistic properties appear with/within a similar set of ‘contexts’ (Harris, 1954). For example, words of related meanings co-occur with similar context words {c1, . . . cn}. This hypothesis implies that if these context words are grouped randomly into m buckets, e.g. {{c1 . . . cx}1, . . . , {cy, . . . cn}m}, then co-related words still co-occur with similar sets of buck- ets. QasemiZadeh and Kallmeyer (2016) exploit this assumption and propose random positive-only projections for building word vectors directly at a reduced dimensionality m. In this paper, we pro- pose a derandomization of this method and a hash- based technique for learning word embeddings. In Section 2, we describe our method. In Section 3, we report results obtained by applying this method to the shared-task benchmark. Finally, we con- clude in Section 4. 2 Method Our method consists of two logical routines: (a) a text skimmer to collect co-occurrence in- formation; and (b) a hash-based encoder to build low-dimensional vectors from collected co- occurrences in (a). Evidently, these procedures can be merged and ordered differently to meet re- quirements of an application. To build an m-dimensional embedding for an entity w (such as a word or phrase) that co-occurs with (or within) some context elements c (result- ing from the skimming routine), we take the fol- lowing steps: Algorithm 1 : Encoding Co-Occurrences 1: ⃗w = ⃗0 2: for each …

Content truncated to 500 words