Context-Dependent Knowledge Graph Embedding

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1656–1661, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Context-Dependent Knowledge Graph Embedding Yuanfei Luo1,2, Quan Wang1∗, Bin Wang1, Li Guo1 1Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China {luoyuanfei,wangquan,wangbin,guoli}@iie.ac.cn 2University of Chinese Academy of Sciences, Beijing, China Abstract We consider the problem of embedding knowledge graphs (KGs) into continuous vector spaces. Existing methods can on- ly deal with explicit relationships within each triple, i.e., local connectivity pattern- s, but cannot handle implicit relationship- s across different triples, i.e., contextual connectivity patterns. This paper proposes context-dependent KG embedding, a two- stage scheme that takes into account both types of connectivity patterns and obtain- s more accurate embeddings. We evaluate our approach on the tasks of link predic- tion and triple classification, and achieve significant and consistent improvements over state-of-the-art methods. 1 Introduction Knowledge Graphs (KGs) like WordNet (Miller, 1995), Freebase (Bollacker et al., 2008), and DB- pedia (Lehmann et al., 2014) have become ex- tremely useful resources for many NLP-related ap- plications. A KG is a directed graph whose nodes correspond to entities and edges to relations. Each edge is a triple of the form (h, r, t), indicating that entities h and t are connected by relation r. Al- though powerful in representing complex data, the symbolic nature makes KGs hard to manipulate. Recently, knowledge graph embedding has at- tracted much attention (Bordes et al., 2011; Bor- des et al., 2013; Socher et al., 2013; Wang et al., 2015). It attempts to embed entities and relations in a KG into a continuous vector space, so as to simplify the manipulation while preserving the in- herent structure of the original graph. Most of the existing KG embedding methods model triples individually, ignoring the fact that ∗Corresponding author: Quan Wang. Shaquille_O_Neal Phoenix_Suns NBA AthletePlaysForTeam TeamPlaysInLeague Nevada USA Utah StateLocatedInCountry StateLocatedInCountry LCPs CCPs Figure 1: LCPs and CCPs. entities connected to a same node are usually im- plicitly related to each other, even if they are not directly connected. Figure 1 gives two examples. Shaquille O Neal and NBA in the former ex- ample and Nevada and Utah in the latter exam- ple are implicitly related to each other, through the intermediate nodes Phoenix Suns and USA re- spectively. We refer to such implicit relationships as contextual connectivity patterns (CCPs). Re- lationships explicitly represented in triples are re- ferred to as local connectivity patterns (LCPs). In most of the existing methods, only LCPs are ex- plicitly modeled. This paper proposes a two-stage embedding scheme that explicitly takes into account both C- CPs and LCPs, called context-dependent KG em- bedding. In the first stage, each CCP is formalized as a knowledge path, i.e., a sequence of entities and relations occurring in the pattern. A word em- bedding model is adopted to learn embeddings of entities and relations, by taking them as pseudo- words. The embeddings are enforced compatible within each knowledge path, and hence can cap- …

Content truncated to 500 words