On the Compositionality Prediction of Noun Phrases using Poincaré Embeddings

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3263–3274 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 3263 On the Compositionality Prediction of Noun Phrases using Poincar´e Embeddings Abhik Jana†, Dmitry Puzyrev‡, Alexander Panchenko⋆, §, Pawan Goyal†, Chris Biemann§, and Animesh Mukherjee† †Indian Institute of Technology Kharagpur, Kharagpur, India ‡National Research University Higher School of Economics, Moscow, Russia ⋆Skolkovo Institute of Science and Technology, Moscow, Russia §Universit¨at Hamburg, Hamburg, Germany abhik.jana@iitkgp.ac.in, {pawang,animeshm}@cse.iitkgp.ac.in dapuzyrev@edu.hse.ru {panchenko,biemann}@informatik.uni-hamburg.de Abstract The compositionality degree of multiword ex- pressions indicates to what extent the meaning of a phrase can be derived from the meaning of its constituents and their grammatical rela- tions. Prediction of (non)-compositionality is a task that has been frequently addressed with distributional semantic models. We introduce a novel technique to blend hierarchical infor- mation with distributional information for pre- dicting compositionality. In particular, we use hypernymy information of the multiword and its constituents encoded in the form of the re- cently introduced Poincar´e embeddings in ad- dition to the distributional information to de- tect compositionality for noun phrases. Us- ing a weighted average of the distributional similarity and a Poincar´e similarity function, we obtain consistent and substantial, statis- tically significant improvement across three gold standard datasets over state-of-the-art models based on distributional information only. Unlike traditional approaches that solely use an unsupervised setting, we have also framed the problem as a supervised task, ob- taining comparable improvements. Further, we publicly release our Poincar´e embeddings, which are trained on the output of handcrafted lexical-syntactic patterns on a large corpus. 1 Introduction An important challenge in Natural Language Pro- cessing is to represent words, phrases, and larger spans in a way that reflects their meaning. Com- positionality is one of the strongest assumptions in semantics, stating that the meaning of larger units can be derived from their smaller parts and their contextual relation. However, for idiomatic phrases, this assumption does not hold true as the meaning of the whole phrase may not be related to their parts in a straightforward fashion. The meaning of the phrases like ‘data format’, ‘head teacher’, ‘green tree’ can easily be understood from the constituent words whereas the semantics of the idiomatic phrases like ‘couch potato’, ‘rat race’, ‘nut case’ are non-compositional, i.e., refer to a different meaning than their parts suggest. In this work, we address compositionality pre- diction, which is the task of assigning a numerical score to a phrase indicating the extent to which the meaning of the phrase can be derived from the meanings of its constituent words. To motivate its importance, e.g., in machine translation, non- compositional phrases must be translated as a unit; in word sense disambiguation, assigning one of the constituent word’s senses to the whole phrase should be avoided for idiomatic phrases; semantic parsing also requires to correctly identify complex predicates and their arguments in this way. A significant amount of effort has gone into operationalizing dense-vector distributional se- mantic models (DSMs) of …

Content truncated to 500 words