Generalized Tuning of Distributional Word Vectors for Monolingual and Cross-Lingual Lexical Entailment

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4824–4830 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 4824 Generalized Tuning of Distributional Word Vectors for Monolingual and Cross-Lingual Lexical Entailment Goran Glavaˇs University of Mannheim Data and Web Science Group B6, 26, DE-68161 Mannheim, Germany goran@informatik.uni-mannheim.de Ivan Vuli´c PolyAI Ltd. 144A Clerkenwell Road London, United Kingdom ivan@poly-ai.com Abstract Lexical entailment (LE; also known as hyponymy-hypernymy or is-a relation) is a core asymmetric lexical relation that supports tasks like taxonomy induction and text generation. In this work, we propose a simple and effec- tive method for fine-tuning distributional word vectors for LE. Our Generalized Lexical EN- tailment model (GLEN) is decoupled from the word embedding model and applicable to any distributional vector space. Yet – unlike exist- ing retrofitting models – it captures a general specialization function allowing for LE-tuning of the entire distributional space and not only the vectors of words seen in lexical constraints. Coupled with a multilingual embedding space, GLEN seamlessly enables cross-lingual LE de- tection. We demonstrate the effectiveness of GLEN in graded LE and report large improve- ments (over 20% in accuracy) over state-of- the-art in cross-lingual LE detection. 1 Background and Motivation Lexical entailment (LE; hyponymy-hypernymy or is-a relation), is a fundamental asymmetric lexico- semantic relation (Collins and Quillian, 1972; Beckwith et al., 1991) and a key building block of lexico-semantic networks and knowledge bases (Fellbaum, 1998; Navigli and Ponzetto, 2012). Rea- soning about word-level entailment supports a mul- titude of tasks such as taxonomy induction (Snow et al., 2006; Navigli et al., 2011; Gupta et al., 2017), natural language inference (Dagan et al., 2013; Bowman et al., 2015; Williams et al., 2018), metaphor detection (Mohler et al., 2013), and text generation (Biran and McKeown, 2013). Due to their distributional nature (Harris, 1954), embedding models (Mikolov et al., 2013; Levy and Goldberg, 2014; Pennington et al., 2014; Melamud et al., 2016; Bojanowski et al., 2017; Peters et al., 2018, inter alia) conflate paradigmatic relations (e.g., synonymy, antonymy, LE, meronymy) and the broader topical (i.e., syntagmatic) relatedness (Schwartz et al., 2015; Mrkˇsi´c et al., 2017). Con- sequently, distributional vectors (i.e., embeddings) cannot be directly used to reliably detect LE. Embedding specialization methods remedy for the semantic vagueness of distributional spaces, forcing the vectors to conform to external linguis- tic constraints (e.g., synonymy or LE word pairs) in order to emphasize the lexico-semantic relation of interest (e.g., semantic similarity of LE) and di- minish the contributions of other types of semantic association. Lexical specialization models gener- ally belong to one of the two families: (1) joint op- timization models and (2) retrofitting (also known as fine-tuning or post-processing) models. Joint models incorporate linguistic constraints directly into the objective of an embedding model, e.g., Skip-Gram (Mikolov et al., 2013), by modifying the prior or regularization of the objective (Yu and Dredze, 2014; Xu et al., 2014; Kiela et al., 2015) or by augmenting the objective with additional factors …

Content truncated to 500 words