Robust Knowledge Graph Completion with Stacked Convolutions and a Student Re-Ranking Network

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 1016–1029 August 1–6, 2021. ©2021 Association for Computational Linguistics 1016 Robust Knowledge Graph Completion with Stacked Convolutions and a Student Re-Ranking Network Justin Lovelace1 Denis Newman-Griffis2 Shikhar Vashishth3∗ Jill Fain Lehman4 Carolyn Penstein Ros´e1 1Language Technologies Institute, Carnegie Mellon University, USA 2Department of Biomedical Informatics, University of Pittsburgh, USA 3Microsoft Research, 4Human-Computer Interaction Institute, Carnegie Mellon University, USA {jlovelac, jfl, cpa3}@cs.cmu.edu, dnewmangriffis@pitt.edu t-svashishth@microsoft.com Abstract Knowledge Graph (KG) completion research usually focuses on densely connected bench- mark datasets that are not representative of real KGs. We curate two KG datasets that include biomedical and encyclopedic knowl- edge and use an existing commonsense KG dataset to explore KG completion in the more realistic setting where dense connectivity is not guaranteed. We develop a deep convolu- tional network that utilizes textual entity rep- resentations and demonstrate that our model outperforms recent KG completion methods in this challenging setting. We find that our model’s performance improvements stem pri- marily from its robustness to sparsity. We then distill the knowledge from the convolutional network into a student network that re-ranks promising candidate entities. This re-ranking stage leads to further improvements in perfor- mance and demonstrates the effectiveness of entity re-ranking for KG completion.1 1 Introduction Knowledge graphs (KGs) have been shown to be useful for a wide range of NLP tasks, such as ques- tion answering (Bordes et al., 2014a,b), dialog sys- tems (Ma et al., 2015), relation extraction (Mintz et al., 2009; Vashishth et al., 2018), and recom- mender systems (Zhang et al., 2016). However, because scaling the collection of facts to provide coverage for all the true relations that hold between entities is difficult, most existing KGs are incom- plete (Dong et al., 2014), limiting their utility for downstream applications. Because of this problem, KG completion (KGC) has come to be a widely studied task (Yang et al., 2015; Trouillon et al., 2016; Shang et al., 2018; Dettmers et al., 2018; ∗ Work performed while at Carnegie Mellon University. 1https://github.com/justinlovelace/ robust-kg-completion Sun et al., 2019; Balazevic et al., 2019; Malaviya et al., 2020; Vashishth et al., 2020a). The increased interest in KGC has led to the curation of a number of benchmark datasets such as FB15K (Bordes et al., 2013), WN18 (Bordes et al., 2013), FB15k-237 (Toutanova and Chen, 2015), and YAGO3-10 (Rebele et al., 2016) that have been the focus of most of the work in this area. However, these benchmark datasets are often curated in such a way as to produce densely connected networks that simplify the task and are not representative of real KGs. For instance, FB15K includes only entities with at least 100 links in Freebase, while YAGO3-10 is limited to only include entities in YAGO3 (Rebele et al., 2016) that have at least 10 relations. Real KGs are not as uniformly dense as these benchmark datasets and have many sparsely con- nected entities (Pujara et …

Content truncated to 500 words