GAKE: Graph Aware Knowledge Embedding

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 641–651, Osaka, Japan, December 11-17 2016. GAKE: Graph Aware Knowledge Embedding Jun Feng⋆, Minlie Huang⋆, Yang Yang†, and Xiaoyan Zhu⋆ ⋆ State Key Lab. on Intelligent Technology and Systems, Tsinghua National Lab. for Information Science and Technology, Dept. of Computer Science and Technology, Tsinghua University † College of Computer Science and Technology, Zhejiang University feng-j13@mails.tsinghua.edu.cn , aihuang@tsinghua.edu.cn yangya@zju.edu.cn , zxy-dcs@tsinghua.edu.cn Abstract Knowledge embedding, which projects triples in a given knowledge base to d-dimensional vec- tors, has attracted considerable research efforts recently. Most existing approaches treat the given knowledge base as a set of triplets, each of whose representation is then learned separately. How- ever, as a fact, triples are connected and depend on each other. In this paper, we propose a graph aware knowledge embedding method (GAKE), which formulates knowledge base as a directed graph, and learns representations for any vertices or edges by leveraging the graph’s structural information. We introduce three types of graph context for embedding: neighbor context, path context, and edge context, each reflects properties of knowledge from different perspectives. We also design an attention mechanism to learn representative power of different vertices or edges. To validate our method, we conduct several experiments on two tasks. Experimental results suggest that our method outperforms several state-of-art knowledge embedding models. 1 Introduction Knowledge bases, such as DBpedia, YAGO, and Freebase, are important resources to store complex structured facts about the real world in the form of triplets as (head entity, relation, tail entity). These knowledge bases have benefited many applications, such as web search and question answer. In the meanwhile, knowledge base embedding, which aims to learn a D-dimensional vector for each subject (i.e., an entity or a relation) in a given knowledge base, has attracted considerable research efforts re- cently (Bordes et al., 2013; Wang et al., 2014; Lin et al., 2015b; Ji et al., 2015). For instance, TransE method (Bordes et al., 2013) regards the relation in a triplet as a translation between the embedding of the two entities. In other words, TransE learns a preference of h + r = t for each triple, where h, r, and t are the representation vector of head entity, relation, and tail entity respectively. Similar ideas are also proposed in TransH (Wang et al., 2014), TransR (Lin et al., 2015b), TransSparse (Ji et al., 2016), etc. Despite the success of above methods in learning knowledge representations, most of them mainly consider knowledge base as a set of triples and models each triple separately and independently. How- ever, in reality, triples are connected to each other and the whole knowledge base could be regarded as a directed graph consisting of vertices (i.e., entities) and directed edges (i.e., relations). In this way, we see that most of existing methods only consider “one hop” information about directed linked entities while miss more global information, such as multiple-steps paths, K-degree neighbors of a given vertex, etc. We call these …

Content truncated to 500 words