KB-LDA: Jointly Learning a Knowledge Base of Hierarchy, Relations, and Facts

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 1449–1459, Beijing, China, July 26-31, 2015. c⃝2015 Association for Computational Linguistics KB-LDA: Jointly Learning a Knowledge Base of Hierarchy, Relations, and Facts Dana Movshovitz-Attias Computer Science Department Carnegie Mellon University dma@cs.cmu.edu William W. Cohen Machine Learning Department Carnegie Mellon University wcohen@cs.cmu.edu Abstract Many existing knowledge bases (KBs), in- cluding Freebase, Yago, and NELL, rely on a fixed ontology, given as an input to the system, which defines the data to be cataloged in the KB, i.e., a hierar- chy of categories and relations between them. The system then extracts facts that match the predefined ontology. We pro- pose an unsupervised model that jointly learns a latent ontological structure of an input corpus, and identifies facts from the corpus that match the learned structure. Our approach combines mixed member- ship stochastic block models and topic models to infer a structure by jointly mod- eling text, a latent concept hierarchy, and latent semantic relationships among the entities mentioned in the text. As a case study, we apply the model to a corpus of Web documents from the software do- main, and evaluate the accuracy of the var- ious components of the learned ontology. 1 Introduction Knowledge base (KB) construction methods can be broadly categorized along several dimensions. One dimension is ontology-guided construction, where the list of categories and relations that de- fine the schema of the KB are explicit, versus open IE methods, where they are not. Another dimen- sion is the type of relations and types included in the KB: some KBs, like WordNet, are hierarchi- cal, in that they contain mainly concept types, su- pertypes and instances, while other KBs contain many types of relationships between concepts. Hi- erarchical knowledge can be learned by methods including distributional clustering (Pereira et al., 1993), as well as Hearst patterns (Hearst, 1992) and similar techniques (Snow et al., 2006). Re- verb (Fader et al., 2011) and TextRunner (Yates et al., 2007) are open methods for learning multi- relation KBs. Finally, NELL (Carlson et al., 2010; Mitchell et al., 2015), FreeBase (Google, 2011) and Yago (Suchanek et al., 2007; Hoffart et al., 2013) are ontology-guided methods for extracting KBs containing both hierarchies and relations. One advantage of ontology-guided methods is that the extracted knowledge is easier to reason with. An advantage of open IE methods is that ontologies may be incomplete, and are expensive to construct for a new domain. Ontology design involves assembling a set of categories, organized in a meaningful hierarchical structure, often pro- viding seeds, i.e., representative examples for each category, and finally, defining inter-category rela- tions. This process is often done manually (Carl- son et al., 2010) leading to a rigid set of categories. Redesigning a new ontology for a specialized do- main represents an additional challenge as it re- quires extensive knowledge of the domain. In this paper, we propose an unsupervised model that learns a …

Content truncated to 500 words