IntKB: A Verifiable Interactive Framework for Knowledge Base Completion

Proceedings of the 28th International Conference on Computational Linguistics, pages 5591–5603 Barcelona, Spain (Online), December 8-13, 2020 5591 IntKB: A Verifiable Interactive Framework for Knowledge Base Completion Bernhard Kratzwald♦ Guo Kunpeng♠♥ Stefan Feuerriegel♦ Dennis Diefenbach♠♥ ♦ Chair of Management Information Systems, ETH Zurich ♠ CNRS, Laboratoire Hubert Curien UMR 5516, University of Lyon ♥ The QA Company SAS, France {bkratzwald, sfeuerriegel}@ethz.ch {kunpeng.guo, dennis.diefenbach}@the-qa-company.com Abstract Knowledge bases (KBs) are essential for many downstream NLP tasks, yet their prime shortcom- ing is that they are often incomplete. State-of-the-art frameworks for KB completion often lack sufficient accuracy to work fully automated without human supervision. As a remedy, we propose IntKB: a novel interactive framework for KB completion from text based on a question answer- ing pipeline. Our framework is tailored to the specific needs of a human-in-the-loop paradigm: (i) We generate facts that are aligned with text snippets and are thus immediately verifiable by humans. (ii) Our system is designed such that it continuously learns during the KB completion task and, therefore, significantly improves its performance upon initial zero- and few-shot rela- tions over time. (iii) We only trigger human interactions when there is enough information for a correct prediction. Therefore, we train our system with negative examples and a fold-option if there is no answer. Our framework yields a favorable performance: it achieves a hit@1 ratio of 29.7% for initially unseen relations, upon which it gradually improves to 46.2%. 1 Introduction Knowledge bases (KBs) present databases that store information about entities and the relations among them. Prominent examples are, for instance, Wikidata (Vrandeˇci´c and Kr¨otzsch, 2014), Freebase (Bol- lacker et al., 2008), NELL (Carlson et al., 2010), DBpedia (Auer et al., 2007), or Yago (Suchanek et al., 2008). Owed to their structured information, KBs are widely used in various downstream tasks of NLP such as, e. g., entity linking (Mendes et al., 2011), query expansion (Dalton et al., 2014), co-reference resolution (Rahman and Ng, 2011), and question answering (Diefenbach et al., 2019). Yet the perfor- mance of downstream NLP tasks is often impeded due to the issue that KBs are incomplete. For example, for newspapers, Wikidata lists the place of publication for only 37% of them, while the owner is stored for only a staggering 9%. Hence, there is a need to develop scalable strategies for KB completion. Prior work on KB completion can be divided into two paradigms. On the one hand, KB completion via link prediction. Here the structural information of an existing, yet incomplete KB is leveraged in order to predict new facts (Lao et al., 2011; Riedel et al., 2013; Neelakantan et al., 2015). On the other hand, KB completion can be built upon a text corpus, so that new facts are extracted directly from the narrative materials (Dong et al., 2014; West et al., 2014). Approaches belonging to the latter paradigm are scarce, as they usually rely upon some way of aggregation information from multiple, different corpora (where credible additions to a KB are signaled by cross-corpus …

Content truncated to 500 words