TransG : A Generative Model for Knowledge Graph Embedding

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 2316–2325, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics TransG : A Generative Model for Knowledge Graph Embedding Han Xiao, Minlie Huang∗, Xiaoyan Zhu State Key Lab. of Intelligent Technology and Systems National Lab. for Information Science and Technology Dept. of Computer Science and Technology Tsinghua University, Beijing 100084, PR China {aihuang, zxy-dcs}@tsinghua.edu.cn Abstract Recently, knowledge graph embedding, which projects symbolic entities and rela- tions into continuous vector space, has be- come a new, hot topic in artificial intelli- gence. This paper proposes a novel gen- erative model (TransG) to address the is- sue of multiple relation semantics that a relation may have multiple meanings re- vealed by the entity pairs associated with the corresponding triples. The new model can discover latent semantics for a rela- tion and leverage a mixture of relation- specific component vectors to embed a fact triple. To the best of our knowledge, this is the first generative model for knowl- edge graph embedding, and at the first time, the issue of multiple relation seman- tics is formally discussed. Extensive ex- periments show that the proposed model achieves substantial improvements against the state-of-the-art baselines. 1 Introduction Abstract or real-world knowledge is always a ma- jor topic in Artificial Intelligence. Knowledge bases such as Wordnet (Miller, 1995) and Free- base (Bollacker et al., 2008) have been shown very useful to AI tasks including question answering, knowledge inference, and so on. However, tra- ditional knowledge bases are symbolic and logic, thus numerical machine learning methods can- not be leveraged to support the computation over the knowledge bases. To this end, knowledge graph embedding has been proposed to project en- tities and relations into continuous vector spaces. Among various embedding models, there is a line ∗ Correspondence author of translation-based models such as TransE (Bor- des et al., 2013), TransH (Wang et al., 2014), TransR (Lin et al., 2015b), and other related mod- els (He et al., 2015) (Lin et al., 2015a). Figure 1: Visualization of TransE embedding vec- tors with PCA dimension reduction. Four relations (a ∼ d) are chosen from Freebase and Wordnet. A dot denotes a triple and its position is decided by the difference vector between tail and head en- tity (t − h). Since TransE adopts the principle of t − h ≈ r, there is supposed to be only one cluster whose centre is the relation vector r. However, re- sults show that there exist multiple clusters, which justifies our multiple relation semantics assump- tion. A fact of knowledge base can usually be rep- resented by a triple (h, r, t) where h, r, t indicate a head entity, a relation, and a tail entity, respec- tively. All translation-based models almost follow the same principle hr + r ≈ tr where hr, r, tr in- 2316 dicate the embedding vectors of triple (h, r, t), with the head and tail entity vector projected with respect to the relation …

Content truncated to 500 words