Aligning Knowledge and Text Embeddings by Entity Descriptions

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 267–272, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Aligning Knowledge and Text Embeddings by Entity Descriptions Huaping Zhong§ , Jianwen Zhang† , Zhen Wang§ , Hai Wan§ , Zheng Chen† §{zhonghp@mail2,wangzh56@mail2,wanhai@mail}.sysu.edu.cn †{jiazhan,zhengc}@microsoft.com §Sun Yat-sen University †Microsoft Research Abstract We study the problem of jointly em- bedding a knowledge base and a text corpus. The key issue is the alignment model making sure the vectors of entities, relations and words are in the same space. Wang et al. (2014a) rely on Wikipedia an- chors, making the applicable scope quite limited. In this paper we propose a new alignment model based on text descrip- tions of entities, without dependency on anchors. We require the embedding vector of an entity not only to fit the structured constraints in KBs but also to be equal to the embedding vector computed from the text description. Extensive experiments show that, the proposed approach consis- tently performs comparably or even better than the method of Wang et al. (2014a), which is encouraging as we do not use any anchor information. 1 Introduction Knowledge base embedding has attracted surging interest recently. The aim is to learn continuous vector representations (embeddings) for entities and relations of a structured knowledge base (KB) such as Freebase. Typically it optimizes a global objective function over all the facts in the KB and hence the embedding vector of an entity / relation is expected to encode global information in the KB. It is capable of reasoning missing facts in a KB and helping facts extraction (Bordes et al., 2011; Bordes et al., 2012; Bordes et al., 2013; Socher et al., 2013; Chang et al., 2013; Wang et al., 2014b; Lin et al., 2015). Although seeming encouraging, the approaches in the aforementioned literature suffer from two common issues: (1) Embeddings are exclusive to entities/relations within KBs. Computation between KBs and text cannot be handled, which are prevalent in practice. For example, in fact extraction, a candidate value may be just a phrase in text. (2) KB sparsity. The above approaches are only based on structured facts of KBs, and thus cannot work well on entities with few facts. An important milestone, the approach of Wang et al. (2014a) solves issue (1) by jointly embed- ding entities, relations, and words into the same vector space and hence is able to deal with word- s/phrases beyond entities in KBs. The key com- ponent is the so-called alignment model, which makes sure the embeddings of entities, relations, and words are in the same space. Two alignment models are introduced there: one uses entity names and another uses Wikipedia anchors. How- ever, both of them have drawbacks. As reported in the paper, using entity names severely pollutes the embeddings of words. Thus it is not recommended in practice. Using Wikipedia anchors completely relies on the special data source and hence the approach cannot be applied to other customer data. To fully address …

Content truncated to 500 words