Predicting the Compositionality of Nominal Compounds: Giving Word Embeddings a Hard Time

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 1986–1997, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Predicting the Compositionality of Nominal Compounds: Giving Word Embeddings a Hard Time Silvio Cordeiro1,2, Carlos Ramisch1, Marco Idiart3, Aline Villavicencio2 1 Aix Marseille Universit´e, CNRS, LIF UMR 7279 (France) 2 Institute of Informatics, Federal University of Rio Grande do Sul (Brazil) 3 Institute of Physics, Federal University of Rio Grande do Sul (Brazil) silvioricardoc@gmail.com carlos.ramisch@lif.univ-mrs.fr marco.idiart@gmail.com avillavicencio@inf.ufrgs.br Abstract Distributional semantic models (DSMs) are often evaluated on artificial simi- larity datasets containing single words or fully compositional phrases. We present a large-scale multilingual eval- uation of DSMs for predicting the de- gree of semantic compositionality of nom- inal compounds on 4 datasets for En- glish and French. We build a total of 816 DSMs and perform 2,856 evaluations using word2vec, GloVe, and PPMI-based models. In addition to the DSMs, we com- pare the impact of different parameters, such as level of corpus preprocessing, con- text window size and number of dimen- sions. The results obtained have a high correlation with human judgments, being comparable to or outperforming the state of the art for some datasets (Spearman’s ρ=.82 for the Reddy dataset). 1 Introduction Distributional semantic models (DSMs) use con- text information to represent the meaning of lexi- cal units as vectors. They normally focus on the accurate semantic representation of single words. It is based on single words that many optimiza- tions for these models have been proposed (Lin, 1999; Erk and Pad´o, 2010; Baroni and Lenci, 2010). This is particularly true for word embed- dings, that is, a type of DSM where distributional vectors are obtained as a by-product of training a neural network to learn a function between words and their contexts (Mikolov et al., 2013a). Simultaneously, there has been intensive re- search on models to compose individual word vec- tors in order to create representations for larger units such as phrases, sentences and even whole documents (Mitchell and Lapata, 2010; Mikolov et al., 2013a). Larger units can often be assumed to have their meanings derived from their parts ac- cording to the language’s grammar, but this is not always the case (Sag et al., 2002). Many multi- word units are associated with idiomatic interpre- tations, unrelated to the meaning of the component words (e.g. silver bullet, eager beaver). Precision-oriented NLP applications need to be able to identify partly-compositional and id- iomatic cases and ensure meaning preservation during processing. Compositionality identifica- tion is a first step towards complete semantic inter- pretation in tasks such as machine translation (to translate non-compositional compounds as a unit), word sense disambiguation (to avoid assigning a sense to parts of non-compositional compounds), and semantic parsing (to identify complex predi- cates and their arguments). Even when larger units are explicitly repre- sented in DSMs (McCarthy et al., 2003; Reddy et al., 2011; Mikolov et al., 2013c; Ferret, 2014), it is not clear whether the quality of these repre- sentations is comparable …

Content truncated to 500 words