Collaborative Policy Learning for Open Knowledge Graph Reasoning

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 2672–2681, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 2672 Collaborative Policy Learning for Open Knowledge Graph Reasoning Cong Fu1∗, Tong Chen2∗, Meng Qu3, Woojeong Jin4, Xiang Ren4 1Zhejiang University, 2Carnegie Mellon University, 3University of Montreal 4University of Southern California fc731097343@gmail.com, tongc2@andrew.cmu.edu, meng.qu@umontreal.ca {woojeong.jin, xiangren}@usc.edu Abstract In recent years, there has been a surge of inter- ests in interpretable graph reasoning methods. However, these models often suffer from lim- ited performance when working on sparse and incomplete graphs, due to the lack of eviden- tial paths that can reach target entities. Here we study open knowledge graph reasoning—a task that aims to reason for missing facts over a graph augmented by a background text corpus. A key challenge of the task is to filter out “ir- relevant” facts extracted from corpus, in order to maintain an effective search space during path inference. We propose a novel reinforce- ment learning framework to train two collab- orative agents jointly, i.e., a multi-hop graph reasoner and a fact extractor. The fact extrac- tion agent generates fact triples from corpora to enrich the graph on the fly; while the reason- ing agent provides feedback to the fact extrac- tor and guides it towards promoting facts that are helpful for the interpretable reasoning. Ex- periments on two public datasets demonstrate the effectiveness of the proposed approach. Source code and datasets used in this paper can be downloaded at https://github. com/shanzhenren/CPL. 1 Introduction Knowledge graph completion or reasoning—i.e., the task of inferring the missing facts (entity re- lationships) for a given graph—is an important problem in natural language processing and has a wide range of applications (Bordes et al., 2011; Socher et al., 2013; Trouillon et al., 2016). Re- cent neural graph reasoning methods, such as MINERVA (Das et al., 2017), DeepPath (Xiong et al., 2017) and Multi-Hop (Lin et al., 2018), have achieved impressive results on the task, of- fering both good prediction accuracy (compared to embedding-based methods (Trouillon et al., ∗ Work done while the authors interned at USC. EU USA B.M.W. USA China Stainless Steel trade_war_with is_main_producer_ of trade_war_with raise_tariff_of trade_war_with ∧ is_main_producer_ of→ raise_tariff_of reasoning summarize pattern ??? is_main_producer_ of raise_tariff_of Figure 1: Illustration of the Knowledge Graph Rea- soning Task. Given an entity (e.g., Miami) and a query relation (e.g., located in), we learn to infer reasoning paths over the existing graph structure to help predict the answer entity (i.e., USA). 2016; Dettmers et al., 2018)) and interpretability of the model predictions. These reasoning meth- ods frame the link inference task as a path finding problem over the graph (see Fig. 1 for example). However, current neural graph reasoning meth- ods encounter two main challenges as follows: (1) their performance are often sensitive to the sparsity and completeness of the graph—missing edges (i.e., potential false positives) make it harder to find evidential paths reaching target …

Content truncated to 500 words