mwetoolkit+sem: Integrating Word Embeddings in the mwetoolkit for Semantic MWE Processing

mwetoolkit+sem: Integrating Word Embeddings in the mwetoolkit for Semantic MWE Processing Silvio Cordeiro1,2, Carlos Ramisch2, Aline Villavicencio1 1 Institute of Informatics, Federal University of Rio Grande do Sul (Brazil) 2 Aix Marseille Université, CNRS, LIF UMR 7279 (France) silvioricardoc@gmail.com carlos.ramisch@lif.univ-mrs.fr avillavicencio@inf.ufrgs.br Abstract This paper presents mwetoolkit+sem: an extension of the mwetoolkit that estimates semantic compositionality scores for multiword expressions (MWEs) based on word embeddings. First, we describe our implementation of vector-space operations working on distributional vectors. The compositionality score is based on the cosine distance between the MWE vector and the composition of the vectors of its member words. Our generic system can handle several types of word embeddings and MWE lists, and may combine individual word representations using several composition techniques. We evaluate our implementation on a dataset of 1042 English noun compounds (Farahmand et al., 2015), comparing different configurations of the underlying word embeddings and word-composition models. We show that our vector-based scores model non-compositionality better than standard association measures such as log-likelihood. Keywords: Lexical semantics, multiword expressions, compositionality, word embeddings. 1. Introduction Multiword expressions (MWEs) are often defined as word combinations “whose exact and unambiguous meaning or connotation cannot be derived directly from the meaning or connotation of its components” (Choueka, 1988). broader definition of MWEs considers that, beyond non-A compositionality, MWEs are word combinations formed by at least 2 lexemes and that present some lexical, syntactic, semantic, pragmatic and/or statistical idiosyncrasy (Bald- win and Kim, 2010). Prototypical examples of MWEs cited in related work are often idioms such as kick the bucket or let the cat out of the bag. With the growing interest in processing MWEs, there is an increasing need for tools that represent their lexical, syntac- tic and semantic characteristics. The mwetoolkit (Ramisch, 2015) provides one such language-independent framework for MWE discovery and identification in corpora. It has been successfully used for modeling lexical and syntactic characteristics of MWEs in many languages. It supports association scores that estimate the degree of conventional- ity of a candidate MWE based on the individual frequencies of component words. However, the ability to model one of the most salient properties of these expressions, that of se- mantic idiosyncrasy, was so far lacking. We now introduce mwetoolkit+sem, containing a new set of capabilities that allow the use of distributional seman- tic models (DSMs) to estimate the semantic composition- ality of MWEs. To this end, we have implemented and tested state-of-the-art techniques for estimating the com- positionality of multiword combinations given the vector representations (or embeddings) of their component words. Our general principle is to combine the individual co- occurrence vectors of words and compare the result to the vector representing the co-occurrence pattern of the whole MWE. If both are close, the MWE is quite compositional, otherwise it is idiomatic. In this paper, we compare standard association scores with the newly implemented models for predicting composi- tionality scores. We show that association scores capture conventional/compositional MWEs while compositionality scores capture idiomaticity. We evaluate mwetoolkit+sem on an existing dataset …

Content truncated to 500 words