Hierarchical Embeddings for Hypernymy Detection and Directionality

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 233–243 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Hierarchical Embeddings for Hypernymy Detection and Directionality Kim Anh Nguyen, Maximilian K¨oper, Sabine Schulte im Walde, Ngoc Thang Vu Institut f¨ur Maschinelle Sprachverarbeitung Universit¨at Stuttgart Pfaffenwaldring 5B, 70569 Stuttgart, Germany {nguyenkh,koepermn,schulte,thangvu}@ims.uni-stuttgart.de Abstract We present a novel neural model HyperVec to learn hierarchical em- beddings for hypernymy detection and directionality. While previous embeddings have shown limitations on prototypical hypernyms, HyperVec represents an unsupervised measure where embeddings are learned in a specific order and capture the hypernym–hyponym distributional hierarchy. Moreover, our model is able to generalize over unseen hypernymy pairs, when using only small sets of training data, and by mapping to other languages. Results on benchmark datasets show that HyperVec outperforms both state-of-the- art unsupervised measures and embedding models on hypernymy detection and directionality, and on predicting graded lexical entailment. 1 Introduction Hypernymy represents a major semantic relation and a key organization principle of semantic mem- ory (Miller and Fellbaum, 1991; Murphy, 2002). It is an asymmetric relation between two terms, a hypernym (superordinate) and a hyponym (subor- diate), as in animal–bird and flower–rose, where the hyponym necessarily implies the hypernym, but not vice versa. From a computational point of view, automatic hypernymy detection is useful for NLP tasks such as taxonomy creation (Snow et al., 2006; Navigli et al., 2011), recognizing tex- tual entailment (Dagan et al., 2013), and text gen- eration (Biran and McKeown, 2013), among many others. Two families of approaches to identify and dis- criminate hypernyms are predominent in NLP, both of them relying on word vector representa- tions. Distributional count approaches make use of either directionally unsupervised measures or of supervised classification methods. Unsupervised measures exploit the distributional inclusion hy- pothesis (Geffet and Dagan, 2005; Zhitomirsky- Geffet and Dagan, 2009), or the distributional informativeness hypothesis (Santus et al., 2014; Rimell, 2014). These measures assign scores to semantic relation pairs, and hypernymy scores are expected to be higher than those of other relation pairs. Typically, Average Precision (AP) (Kotler- man et al., 2010) is applied to rank and distinguish between the predicted relations. Supervised clas- sification methods represent each pair of words as a single vector, by using the concatenation or the element-wise difference of their vectors (Ba- roni et al., 2012; Roller et al., 2014; Weeds et al., 2014). The resulting vector is fed into a Sup- port Vector Machine (SVM) or into Logistic Re- gression (LR), to predict hypernymy. Across ap- proaches, Shwartz et al. (2017) demonstrated that there is no single unsupervised measure which consistently deals well with discriminating hyper- nymy from other semantic relations. Furthermore, Levy et al. (2015) showed that supervised meth- ods memorize prototypical hypernyms instead of learning a relation between two words. Approaches of hypernymy-specific embed- dings utilize neural models to learn vector rep- resentations for hypernymy. Yu et al. (2015) proposed a supervised method to learn term em- beddings for hypernymy identification, based on pre-extracted …

Content truncated to 500 words