Representing Text for Joint Embedding of Text and Knowledge Bases

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1499–1509, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Representing Text for Joint Embedding of Text and Knowledge Bases Kristina Toutanova Microsoft Research Redmond, WA, USA Danqi Chen∗ Computer Science Department Stanford University Patrick Pantel Microsoft Research Redmond, WA, USA Hoifung Poon Microsoft Research Redmond, WA, USA Pallavi Choudhury Microsoft Research Redmond, WA, USA Michael Gamon Microsoft Research Redmond, WA, USA Abstract Models that learn to represent textual and knowledge base relations in the same con- tinuous latent space are able to perform joint inferences among the two kinds of re- lations and obtain high accuracy on knowl- edge base completion (Riedel et al., 2013). In this paper we propose a model that cap- tures the compositional structure of tex- tual relations, and jointly optimizes entity, knowledge base, and textual relation rep- resentations. The proposed model signifi- cantly improves performance over a model that does not share parameters among tex- tual relations with common sub-structure. 1 Introduction Representing information about real-world enti- ties and their relations in structured knowledge base (KB) form enables numerous applications. Large, collaboratively created knowledge bases have recently become available e.g., Freebase (Bollacker et al., 2008), YAGO (Suchanek et al., 2007), and DBPedia (Auer et al., 2007), but even though they are impressively large, their coverage is far from complete. This has motivated research in automatically deriving new facts to extend a manually built knowledge base, by using infor- mation from the existing knowledge base, textual mentions of entities, and semi-structured data such as tables and web forms (Nickel et al., 2015). In this paper we build upon the work of Riedel et al. (2013), which jointly learns continuous rep- resentations for knowledge base and textual rela- tions. This common representation in the same vector space can serve as a kind of “universal schema” which admits joint inferences among ∗This research was conducted during the author’s intern- ship at Microsoft Research. Knowledge Base Barack Obama United States Honolulu Textual Mentions Barack Obama is the 44th and current President of United States. Obama was born in the United States just as he has always said. … ClueWeb place_of_birth city_of nationality Figure 1: A knowledge base fragment coupled with textual mentions of pairs of entities. KBs and text. The textual relations represent the relationships between entities expressed in indi- vidual sentences (see Figure 1 for an example). Riedel et al. (2013) represented each textual men- tion of an entity pair by the lexicalized depen- dency path between the two entities (see Figure 2). Each such path is treated as a separate relation in a combined knowledge graph including both KB and textual relations. Following prior work in la- tent feature models for knowledge base comple- tion, every textual relation receives its own contin- uous representation, learned from the pattern of its co-occurrences in the knowledge graph. However, largely synonymous textual relations often share common sub-structure, and are com- posed of similar words and …

Content truncated to 500 words