Semantically Smooth Knowledge Graph Embedding

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 84–94, Beijing, China, July 26-31, 2015. c⃝2015 Association for Computational Linguistics Semantically Smooth Knowledge Graph Embedding Shu Guo†, Quan Wang†∗, Bin Wang†, Lihong Wang‡, Li Guo† †Institute of Information Engineering, Chinese Academy of Sciences, Beijing 100093, China {guoshu,wangquan,wangbin,guoli}@iie.ac.cn ‡National Computer Network Emergency Response Technical Team Coordination Center of China, Beijing 100029, China wlh@isc.org.cn Abstract This paper considers the problem of em- bedding Knowledge Graphs (KGs) con- sisting of entities and relations into low- dimensional vector spaces. Most of the existing methods perform this task based solely on observed facts. The only re- quirement is that the learned embeddings should be compatible within each individ- ual fact. In this paper, aiming at further discovering the intrinsic geometric struc- ture of the embedding space, we propose Semantically Smooth Embedding (SSE). The key idea of SSE is to take full ad- vantage of additional semantic informa- tion and enforce the embedding space to be semantically smooth, i.e., entities be- longing to the same semantic category will lie close to each other in the embedding s- pace. Two manifold learning algorithms Laplacian Eigenmaps and Locally Linear Embedding are used to model the smooth- ness assumption. Both are formulated as geometrically based regularization terms to constrain the embedding task. We em- pirically evaluate SSE in two benchmark tasks of link prediction and triple classi- fication, and achieve significant and con- sistent improvements over state-of-the-art methods. Furthermore, SSE is a general framework. The smoothness assumption can be imposed to a wide variety of em- bedding models, and it can also be con- structed using other information besides entities’ semantic categories. 1 Introduction Knowledge Graphs (KGs) like WordNet (Miller, 1995), Freebase (Bollacker et al., 2008), and DB- ∗ Corresponding author: Quan Wang. pedia (Lehmann et al., 2014) have become ex- tremely useful resources for many NLP relat- ed applications, such as word sense disambigua- tion (Agirre et al., 2014), named entity recogni- tion (Magnini et al., 2002), and information ex- traction (Hoffmann et al., 2011). A KG is a multi- relational directed graph composed of entities as nodes and relations as edges. Each edge is repre- sented as a triple of fact ⟨ei, rk, e j⟩, indicating that head entity ei and tail entity e j are connected by re- lation rk. Although powerful in representing struc- tured data, the underlying symbolic nature makes KGs hard to manipulate. Recently a new research direction called knowl- edge graph embedding has attracted much atten- tion (Socher et al., 2013; Bordes et al., 2013; Bor- des et al., 2014; Lin et al., 2015). It attempts to embed components of a KG into continuous vector spaces, so as to simplify the manipulation while preserving the inherent structure of the original graph. Specifically, given a KG, entities and re- lations are first represented in a low-dimensional vector space, and for each triple, a scoring func- tion is defined to …

Content truncated to 500 words