SphereRE: Distinguishing Lexical Relations with Hyperspherical Relation Embeddings

SphereRE: Distinguishing Lexical Relations with Hyperspherical Relation Embeddings Chengyu Wang1, Xiaofeng He1∗, Aoying Zhou2 1School of Computer Science and Software Engineering, East China Normal University 2School of Data Science and Engineering, East China Normal University chywang2013@gmail.com, xfhe@sei.ecnu.edu.cn ayzhou@dase.ecnu.edu.cn Abstract Lexical relations describe how meanings of terms relate to each other. Typical relations in- clude hypernymy, synonymy, meronymy, etc. Automatic distinction of lexical relations is vital for NLP applications, and is also chal- lenging due to the lack of contextual signals to discriminate between such relations. In this work, we present a neural representation learning model to distinguish lexical relations among term pairs based on Hyperspherical Re- lation Embeddings (SphereRE). Rather than learning embeddings for individual terms, the model learns representations of relation triples by mapping them to the hyperspherical em- bedding space, where relation triples of dif- ferent lexical relations are well separated. We further introduce a Monte-Carlo based sam- pling and learning algorithm to train the model via transductive learning. Experiments over several benchmarks confirm SphereRE outper- forms state-of-the-arts. 1 Introduction Lexical relations are relations between terms in lexicons. Types of lexical relations include hyper- nymy, synonymy, meronymy, etc. Such relations are treated as key resources for various NLP ap- plications, e.g., question answering (Yang et al., 2017), taxonomy induction (Shen et al., 2018), machine translation (Zhang et al., 2018), natural language inference (Inkpen et al., 2018), lexical database construction (Speer et al., 2017), etc. Due to its importance, automatic acquisition of lexical relations is a research focus in NLP. In early years, lexical relations in WordNet were manually compiled by linguists (Miller, 1995). Recently, path-based and distributional approaches are two major paradigms to classify a term pair into a fixed inventory of lexical re- lations, or to predict it as random (meaning the ∗Corresponding author. two terms are un-related) (Shwartz and Dagan, 2016; Wang et al., 2017a). Path-based approaches use dependency paths connecting two terms to infer lexical relations (Washio and Kato, 2018a; Roller et al., 2018). The paths usually describe relations between terms explicitly, but require the two terms co-occur in a sentence, leading to the “low coverage” problem. Apart from Hearst pat- terns (Hearst, 1992), there are few high-quality textual patterns to recognize lexical relations other than hypernymy. Distributional approaches con- sider the global contexts of terms to predict lexi- cal relations using word embeddings (Baroni et al., 2012; Glavas and Vulic, 2018). They are reported to outperform several path-based approaches, but can suffer from the “lexical memorization” prob- lem (Levy et al., 2015; Shwartz and Dagan, 2016). This is because some supervised distributional ap- proaches learn properties of two terms separately, instead of how two terms relate to each other in the embedding space. (a) Term Embedding Space (b) Relation Embedding Space Car Auto Automobile Vehicle Engine Wheel Hypernymy Synonymy Meronymy (Car, Hypernymy, Vehicle) (Car, Meronymy, Engine) (Car, Meronymy, Wheel) (Car, Synonymy, Auto) (Car, Synonymy, Automobile) O O x x y y Figure 1: An example of hyperspherical learning w.r.t. the term car and three …

Content truncated to 500 words