Modeling Relation Paths for Representation Learning of Knowledge Bases

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 705–714, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Modeling Relation Paths for Representation Learning of Knowledge Bases Yankai Lin1, Zhiyuan Liu1 ∗, Huanbo Luan1, Maosong Sun1, Siwei Rao2, Song Liu2 1 Department of Computer Science and Technology, State Key Lab on Intelligent Technology and Systems, National Lab for Information Science and Technology, Tsinghua University, Beijing, China 2 Samsung R&D Institute of China, Beijing, China Abstract Representation learning of knowledge bases aims to embed both entities and relations into a low-dimensional space. Most existing methods only consider direct relations in representation learning. We argue that multiple-step relation paths also contain rich inference patterns be- tween entities, and propose a path-based representation learning model. This model considers relation paths as translations between entities for representation learn- ing, and addresses two key challenges: (1) Since not all relation paths are reliable, we design a path-constraint resource allo- cation algorithm to measure the reliability of relation paths. (2) We represent relation paths via semantic composition of relation embeddings. Experimental results on real-world datasets show that, as com- pared with baselines, our model achieves significant and consistent improvements on knowledge base completion and re- lation extraction from text. The source code of this paper can be obtained from https://github.com/mrlyk423/ relation_extraction. 1 Introduction People have recently built many large-scale knowledge bases (KBs) such as Freebase, DBpe- dia and YAGO. These KBs consist of facts about the real world, mostly in the form of triples, e.g., (Steve Jobs, FounderOf, Apple Inc.). KBs are important resources for many applications such as question answering and Web search. Although typical KBs are large in size, usually containing thousands of relation types, millions of entities and billions of facts (triples), they are far from ∗Corresponding author: Z. Liu (liuzy@tsinghua.edu.cn) complete. Hence, many efforts have been invested in relation extraction to enrich KBs. Recent studies reveal that, neural-based repre- sentation learning methods are scalable and ef- fective to encode relational knowledge with low- dimensional representations of both entities and relations, which can be further used to extract unknown relational facts. TransE (Bordes et al., 2013) is a typical method in the neural-based ap- proach, which learns vectors (i.e., embeddings) for both entities and relations. The basic idea behind TransE is that, the relationship between two enti- ties corresponds to a translation between the em- beddings of the entities, that is, h + r ≈ t when the triple (h, r, t) holds. Since TransE has issues when modeling 1-to-N, N-to-1 and N-to-N rela- tions, various methods such as TransH (Wang et al., 2014) and TransR (Lin et al., 2015) are pro- posed to assign an entity with different represen- tations when involved in various relations. Despite their success in modeling relational facts, TransE and its extensions only take di- rect relations between entities into considera- tion. It is known that there are also substan- tial multiple-step relation paths between entities indicating their semantic relationships. The re- …

Content truncated to 500 words