A Capsule Network-based Embedding Model for Knowledge Graph Completion and Search Personalization

Proceedings of NAACL-HLT 2019, pages 2180–2189 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2180 A Capsule Network-based Embedding Model for Knowledge Graph Completion and Search Personalization Dai Quoc Nguyen1, Thanh Vu2, Tu Dinh Nguyen1, Dat Quoc Nguyen3, Dinh Phung1 1Monash University, Australia; 3The University of Melbourne, Australia 2The Australian e-Health Research Centre, CSIRO, Australia 1{dai.nguyen,tu.dinh.nguyen,dinh.phung}@monash.edu 2thanh.vu@csiro.au; 3dqnguyen@unimelb.edu.au Abstract In this paper, we introduce an embedding model, named CapsE, exploring a capsule net- work to model relationship triples (subject, re- lation, object). Our CapsE represents each triple as a 3-column matrix where each col- umn vector represents the embedding of an element in the triple. This 3-column matrix is then fed to a convolution layer where mul- tiple filters are operated to generate different feature maps. These feature maps are recon- structed into corresponding capsules which are then routed to another capsule to produce a continuous vector. The length of this vector is used to measure the plausibility score of the triple. Our proposed CapsE obtains better performance than previous state-of-the-art em- bedding models for knowledge graph comple- tion on two benchmark datasets WN18RR and FB15k-237, and outperforms strong search personalization baselines on SEARCH17. 1 Introduction Knowledge graphs (KGs) containing relationship triples (subject, relation, object), denoted as (s, r, o), are the useful resources for many NLP and especially information retrieval applications such as semantic search and question answering (Wang et al., 2017). However, large knowledge graphs, even containing billions of triples, are still incom- plete, i.e., missing a lot of valid triples (West et al., 2014). Therefore, much research efforts have fo- cused on the knowledge graph completion task which aims to predict missing triples in KGs, i.e., predicting whether a triple not in KGs is likely to be valid or not (Bordes et al., 2011, 2013; Socher et al., 2013). To this end, many embedding models have been proposed to learn vector representations for entities (i.e., subject/head entity and object/tail entity) and relations in KGs, and obtained state- of-the-art results as summarized by Nickel et al. (2016a) and Nguyen (2017). These embedding models score triples (s, r, o), such that valid triples have higher plausibility scores than invalid ones (Bordes et al., 2011, 2013; Socher et al., 2013). For example, in the context of KGs, the score for (Melbourne, cityOf, Australia) is higher than the score for (Melbourne, cityOf, United Kingdom). Triple modeling is applied not only to the KG completion, but also for other tasks which can be formulated as a triple-based prediction prob- lem. An example is in search personalization, one would aim to tailor search results to each spe- cific user based on the user’s personal interests and preferences (Teevan et al., 2005, 2009; Ben- nett et al., 2012; Harvey et al., 2013; Vu et al., 2015, 2017). Here the triples can be formulated as (submitted query, user profile, returned document) and used to re-rank documents returned to a user given an input query, by employing an existing KG embedding …

Content truncated to 500 words