Neighborhood Mixture Model for Knowledge Base Completion

Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning (CoNLL), pages 40–50, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Neighborhood Mixture Model for Knowledge Base Completion Dat Quoc Nguyen1, Kairit Sirts1, Lizhen Qu2 and Mark Johnson1 1 Department of Computing, Macquarie University, Sydney, Australia dat.nguyen@students.mq.edu.au, {kairit.sirts, mark.johnson}@mq.edu.au 2 Data61 & Australian National University lizhen.qu@data61.csiro.au Abstract Knowledge bases are useful resources for many natural language processing tasks, however, they are far from complete. In this paper, we define a novel entity rep- resentation as a mixture of its neighbor- hood in the knowledge base and apply this technique on TransE—a well-known embedding model for knowledge base completion. Experimental results show that the neighborhood information signif- icantly helps to improve the results of the TransE, leading to better performance than obtained by other state-of-the-art embed- ding models on three benchmark datasets for triple classification, entity prediction and relation prediction tasks. Keywords: Knowledge base completion, embedding model, mixture model, link prediction, triple classification, entity pre- diction, relation prediction. 1 Introduction Knowledge bases (KBs), such as WordNet (Miller, 1995), YAGO (Suchanek et al., 2007), Freebase (Bollacker et al., 2008) and DBpedia (Lehmann et al., 2015), represent relationships between enti- ties as triples (head entity, relation, tail entity). Even very large knowledge bases are still far from complete (Socher et al., 2013; West et al., 2014). Knowledge base completion or link prediction sys- tems (Nickel et al., 2015) predict which triples not in a knowledge base are likely to be true (Taskar et al., 2004; Bordes et al., 2011). Embedding models for KB completion associate entities and/or relations with dense feature vectors or matrices. Such models obtain state-of-the-art performance (Bordes et al., 2012; Bordes et al., 2013; Socher et al., 2013; Wang et al., 2014; Guu et al., 2015; Nguyen et al., 2016) and generalize to large KBs (Krompa et al., 2015). Most embedding models for KB completion learn only from triples and by doing so, ignore lots of information implicitly provided by the structure of the knowledge graph. Recently, several authors have addressed this issue by incorporating rela- tion path information into model learning (Garc´ıa- Dur´an et al., 2015; Lin et al., 2015a; Guu et al., 2015; Toutanova et al., 2016) and have shown that the relation paths between entities in KBs provide useful information and improve knowledge base completion. For instance, a three-relation path (head, born in hospital/r1, e1) ⇒(e1, hospital located in city/r2, e2) ⇒(e2, city in country/r3, tail) is likely to indicate that the fact (head, nationality, tail) could be true, so the relation path here p = {r1, r2, r3} is useful for predicting the relationship “nationality” between the head and tail entities. Besides the relation paths, there could be other useful information implicitly presented in the knowledge base that could be exploited for better KB completion. For instance, the whole neigh- borhood of entities could provide lots of useful in- formation for predicting the relationship between two entities. Consider for example a KB fragment …

Content truncated to 500 words