An Interpretable Knowledge Transfer Model for Knowledge Base Completion

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 950–962 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1088 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 950–962 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1088 An Interpretable Knowledge Transfer Model for Knowledge Base Completion Qizhe Xie, Xuezhe Ma, Zihang Dai, Eduard Hovy Language Technologies Institute Carnegie Mellon University Pittsburgh, PA 15213, USA {qzxie, xuezhem, dzihang, hovy}@cs.cmu.edu Abstract Knowledge bases are important resources for a variety of natural language process- ing tasks but suffer from incompleteness. We propose a novel embedding model, ITransF, to perform knowledge base com- pletion. Equipped with a sparse atten- tion mechanism, ITransF discovers hidden concepts of relations and transfer statisti- cal strength through the sharing of con- cepts. Moreover, the learned associations between relations and concepts, which are represented by sparse attention vec- tors, can be interpreted easily. We evalu- ate ITransF on two benchmark datasets— WN18 and FB15k for knowledge base completion and obtains improvements on both the mean rank and Hits@10 metrics, over all baselines that do not use additional information. 1 Introduction Knowledge bases (KB), such as WordNet (Fell- baum, 1998), Freebase (Bollacker et al., 2008), YAGO (Suchanek et al., 2007) and DBpe- dia (Lehmann et al., 2015), are useful resources for many applications such as question answer- ing (Berant et al., 2013; Yih et al., 2015; Dai et al., 2016) and information extraction (Mintz et al., 2009). However, knowledge bases suf- fer from incompleteness despite their formidable sizes (Socher et al., 2013; West et al., 2014), lead- ing to a number of studies on automatic knowl- edge base completion (KBC) (Nickel et al., 2015) or link prediction. The fundamental motivation behind these stud- ies is that there exist some statistical regularities under the intertwined facts stored in the multi- relational knowledge base. By discovering gener- alizable regularities in known facts, missing ones may be recovered in a faithful way. Due to its ex- cellent generalization capability, distributed repre- sentations, a.k.a. embeddings, have been popular- ized to address the KBC task (Nickel et al., 2011; Bordes et al., 2011, 2014, 2013; Socher et al., 2013; Wang et al., 2014; Guu et al., 2015; Nguyen et al., 2016b). As a seminal work, Bordes et al. (2013) pro- poses the TransE, which models the statistical regularities with linear translations between en- tity embeddings operated by a relation embed- ding. Implicitly, TransE assumes both entity em- beddings and relation embeddings dwell in the same vector space, posing an unnecessarily strong prior. To relax this requirement, a variety of mod- els first project the entity embeddings to a relation- dependent space (Bordes et al., 2014; Ji et al., 2015; Lin et al., 2015b; Nguyen et al., 2016b), and then model the translation property in the pro- jected space. Typically, these relation-dependent spaces are characterized by the projection matri- ces unique to each relation. As a benefit, …

Content truncated to 500 words