Specialising Word Vectors for Lexical Entailment

Proceedings of NAACL-HLT 2018, pages 1134–1145 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Specialising Word Vectors for Lexical Entailment Ivan Vuli´c1 and Nikola Mrkši´c2 1 Language Technology Lab, University of Cambridge iv250@cam.ac.uk2 PolyAI nikola@poly-ai.com Abstract We present LEAR (Lexical Entailment Attract- Repel), a novel post-processing method that transforms any input word vector space to emphasise the asymmetric relation of lexical entailment (LE), also known as the IS-A or hyponymy-hypernymy relation. By injecting external linguistic constraints (e.g., WordNet links) into the initial vector space, the LE spe- cialisation procedure brings true hyponymy- hypernymy pairs closer together in the trans- formed Euclidean space. The proposed asym- metric distance measure adjusts the norms of word vectors to reflect the actual WordNet- style hierarchy of concepts. Simultaneously, a joint objective enforces semantic similarity us- ing the symmetric cosine distance, yielding a vector space specialised for both lexical re- lations at once. LEAR specialisation achieves state-of-the-art performance in the tasks of hy- pernymy directionality, hypernymy detection, and graded lexical entailment, demonstrating the effectiveness and robustness of the pro- posed asymmetric specialisation model. 1 Introduction Word representation learning has become a re- search area of central importance in NLP, with its usefulness demonstrated across application ar- eas such as parsing (Chen and Manning, 2014), machine translation (Zou et al., 2013), and many others (Turian et al., 2010; Collobert et al., 2011). Standard techniques for inducing word embeddings rely on the distributional hypothesis (Harris, 1954), using co-occurrence information from large textual corpora to learn meaningful word representations (Mikolov et al., 2013; Levy and Goldberg, 2014; Pennington et al., 2014; Bojanowski et al., 2017). A major drawback of the distributional hypoth- esis is that it coalesces different relationships be- tween words, such as synonymy and topical related- ness, into a single vector space. A popular solution is to go beyond stand-alone unsupervised learning and fine-tune distributional vector spaces by using external knowledge from human- or automatically- constructed knowledge bases. This is often done as a post-processing step, where distributional vectors are gradually refined to satisfy linguistic constraints extracted from lexical resources such as WordNet (Faruqui et al., 2015; Mrkši´c et al., 2016), the Para- phrase Database (PPDB) (Wieting et al., 2015), or BabelNet (Mrkši´c et al., 2017; Vuli´c et al., 2017a). One advantage of post-processing methods is that they treat the input vector space as a black box, making them applicable to any input space. A key property of these methods is their abil- ity to transform the vector space by specialising it for a particular relationship between words.1 Prior work has predominantly focused on distinguishing between semantic similarity and conceptual relat- edness (Faruqui et al., 2015; Mrkši´c et al., 2017; Vuli´c et al., 2017b). In this paper, we introduce a novel post-processing model which specialises vec- tor spaces for the lexical entailment (LE) relation. Word-level lexical entailment is an asymmet- ric semantic relation (Collins and Quillian, 1972; Beckwith et al., 1991). It is a key principle de- termining the organisation of …

Content truncated to 500 words