Transition-based Knowledge Graph Embedding with Relational Mapping Properties

PACLIC 28 !328 Transition-based Knowledge Graph Embedding with Relational Mapping Properties Miao Fan†,⇤, Qiang Zhou†, Emily Chang‡, Thomas Fang Zheng†,⇧ †CSLT, Tsinghua National Laboratory for Information Science and Technology, Department of Computer Science and Technology, Tsinghua University, Beijing, 100084, China. ‡Emory University, U.S.A. ⇤fanmiao.cslt.thu@gmail.com, ⇧fzheng@tsinghua.edu.cn Abstract Many knowledge repositories nowadays con- tain billions of triplets, i.e. (head-entity, re- lationship, tail-entity), as relation instances. These triplets form a directed graph with enti- ties as nodes and relationships as edges. How- ever, this kind of symbolic and discrete stor- age structure makes it difficult for us to exploit the knowledge to enhance other intelligence- acquired applications (e.g. the Question- Answering System), as many AI-related al- gorithms prefer conducting computation on continuous data. Therefore, a series of e- merging approaches have been proposed to facilitate knowledge computing via encoding the knowledge graph into a low-dimensional embedding space. TransE is the latest and most promising approach among them, and can achieve a higher performance with few- er parameters by modeling the relationship as a transitional vector from the head entity to the tail entity. Unfortunately, it is not flex- ible enough to tackle well with the various mapping properties of triplets, even though it- s authors spot the harm on performance. In this paper, we thus propose a superior model called TransM to leverage the structure of the knowledge graph via pre-calculating the dis- tinct weight for each training triplet according to its relational mapping property. In this way, the optimal function deals with each triplet de- pending on its own weight. We carry out ex- tensive experiments to compare TransM with the state-of-the-art method TransE and other prior arts. The performance of each approach is evaluated within two different application s- cenarios on several benchmark datasets. Re- sults show that the model we proposed signifi- cantly outperforms the former ones with lower parameter complexity as TransE. 1 Introduction Many knowledge repositories have been construct- ed either by experts with long-term funding (e.g. WordNet1 and OpenCyc2) or by crowds with collab- orative contribution (e.g. Freebase3 and DBpedia4). Most of them store billions of triplets. Each triplet, abbreviated as (h, r, t), is composed by two entities (i.e the head entity h and the tail entity t), and the re- lationship r between them. These triplets can form a huge directed graph for each knowledge repository with millions of entities as nodes and thousands of relationships as edges. Ideally, we can take advantages of these knowl- edge graphs to enhance many other intelligence- dependent systems, such as Information Retrieval Systems (Wical, 1999; Wical, 2000), Question- Answering Systems (Pazzani and Engelman, 1983; Rinaldi et al., 2003; Hermjakob et al., 2000), etc. However, the graph-based knowledge representation is some kind of rigid. More specifically, this sym- bolic and discrete storage structure makes it hard for us to exploit great knowledge treasures, as many AI- related algorithms prefer conducting computations on continuous data. Some recent literatures on Lan- guage Modeling by means of learning distributed word representation (Bengio …

Content truncated to 500 words