Recurrent One-Hop Predictions for Reasoning over Knowledge Graphs

Recurrent One-Hop Predictions for Reasoning over Knowledge Graphs Wenpeng Yin1, Yadollah Yaghoobzadeh2, Hinrich Sch¨utze3 1Dept. of Computer Science, University of Pennsylvania, USA 2 Microsoft Research, Montreal, Canada 3CIS, LMU Munich, Germany wenpeng@seas.upenn.edu Abstract Large scale knowledge graphs (KGs) such as Freebase are generally incomplete. Reasoning over multi-hop (mh) KG paths is thus an important capability that is needed for question answering or other NLP tasks that require knowledge about the world. mh-KG reasoning includes diverse scenarios, e.g., given a head entity and a relation path, predict the tail entity; or given two enti- ties connected by some relation paths, predict the unknown relation between them. We present ROPs, recurrent one-hop predictors, that predict entities at each step of mh-KB paths by using recurrent neural networks and vector representations of entities and relations, with two benefits: (i) modeling mh-paths of arbitrary lengths while updating the entity and relation representations by the training signal at each step; (ii) handling different types of mh-KG reasoning in a unified framework. Our models show state-of-the-art for two important multi-hop KG reasoning tasks: Knowledge Base Completion and Path Query Answering.1 1 Introduction Natural language understanding (NLU) is impossible without knowledge about the world. Large scale knowledge graphs (KGs) such as Freebase (Bollacker et al., 2008) are structures that store world knowl- edge. Unfortunately, KGs suffer from incomplete coverage (Min et al., 2013); e.g., Freebase contains Brandon Lee, but not his ethnicity. The knowledge in KGs needs to be expanded to cover more facts; reasoning is one way to do so. For example, we could infer that (Microsoft, ?, United States) instantiates “CountryOfHQ” given the facts (Microsoft, IsBasedIn, Seattle) and (Seattle, LocatedIn, United States); or we could infer Brandon Lee’s ethnicity from his parents’ ethnicity, i.e., answering the query (Brandon Lee, Ethnicity, ?) by facts (Brandon Lee, Father, Bruce Lee) and (Bruce Lee, Ethnicity, Chinese). We refer to the two reasoning examples as “knowledge base completion (KBC)” and “path query answering (PQA)”, respectively. The most successful approach for modeling KGs is the embedding approach. It embeds KG elements (entities and relations) into low-dimensional dense vectors; controlling the dimensionality of the vector space forces generalization to new facts (Nickel et al., 2011). In this work, we are mainly interested in three issues. (i) Compared to modeling one-hop KG paths, a bigger challenge is how to model multi-hop paths, e.g., the path query (U.S.A, president→spouse→born in, ?) for the question “Where was the first lady of the United States born?” (ii) How can we address different KG reasoning problems driven by multi-hop paths in a universal paradigm rather than via different systems? (iii) How can we combine specific multi-hop KG reasoning tasks with generic KG representation learning, so that KG representation learning can either stand alone or be incorporated into diverse multi-hop KG-related NLU problems. We get inspiration from following two types of work. First, prior work in KG reasoning. Guu et al. (2015) extend one-hop reasoning regimes such as TransE (Bordes et al., 2013) to multi-hop PQA. However, …

Content truncated to 500 words