Investigating Robustness and Interpretability of Link Prediction via Adversarial Modifications

Proceedings of NAACL-HLT 2019, pages 3336–3347 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 3336 Investigating Robustness and Interpretability of Link Prediction via Adversarial Modifications Pouya Pezeshkpour University of California Irvine, CA pezeshkp@uci.edu Yifan Tian University of California Irvine, CA yifant@uci.edu Sameer Singh University of California Irvine, CA sameer@uci.edu Abstract Representing entities and relations in an em- bedding space is a well-studied approach for machine learning on relational data. Existing approaches, however, primarily focus on im- proving accuracy and overlook other aspects such as robustness and interpretability. In this paper, we propose adversarial modifica- tions for link prediction models: identifying the fact to add into or remove from the knowl- edge graph that changes the prediction for a target fact after the model is retrained. Us- ing these single modifications of the graph, we identify the most influential fact for a pre- dicted link and evaluate the sensitivity of the model to the addition of fake facts. We in- troduce an efficient approach to estimate the effect of such modifications by approximating the change in the embeddings when the knowl- edge graph changes. To avoid the combinato- rial search over all possible facts, we train a network to decode embeddings to their corre- sponding graph components, allowing the use of gradient-based optimization to identify the adversarial modification. We use these tech- niques to evaluate the robustness of link predic- tion models (by measuring sensitivity to addi- tional facts), study interpretability through the facts most responsible for predictions (by iden- tifying the most influential neighbors), and de- tect incorrect facts in the knowledge base. 1 Introduction Knowledge graphs (KG) play a critical role in many real-world applications such as search, structured data management, recommendations, and question answering. Since KGs often suffer from incom- pleteness and noise in their facts (links), a number of recent techniques have proposed models that em- bed each entity and relation into a vector space, and use these embeddings to predict facts. These dense representation models for link prediction include tensor factorization [Nickel et al., 2011, Socher et al., 2013, Yang et al., 2015], algebraic opera- tions [Bordes et al., 2011, 2013b, Dasgupta et al., 2018], multiple embeddings [Wang et al., 2014, Lin et al., 2015, Ji et al., 2015, Zhang et al., 2018], and complex neural models [Dettmers et al., 2018, Nguyen et al., 2018]. However, there are only a few studies [Kadlec et al., 2017, Sharma et al., 2018] that investigate the quality of the different KG mod- els. There is a need to go beyond just the accuracy on link prediction, and instead focus on whether these representations are robust and stable, and what facts they make use of for their predictions. In this paper, our goal is to design approaches that minimally change the graph structure such that the prediction of a target fact changes the most after the embeddings are relearned, which we collectively call Completion Robustness and Inter- pretability via Adversarial Graph Edits (CRIAGE). First, we …

Content truncated to 500 words