Adaptive Convolution for Multi-Relational Learning

Proceedings of NAACL-HLT 2019, pages 978–987 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 978 Adaptive Convolution for Multi-Relational Learning Xiaotian Jiang1, Quan Wang2, Bin Wang3 1Cloud and Smart Industries Group, Tencent 2Baidu Inc. 3Xiaomi AI Lab johnxtjiang@tencent.com, quanwang1012@gmail.com, wangbin11@xiaomi.com Abstract We consider the problem of learning dis- tributed representations for entities and rela- tions of multi-relational data so as to predict missing links therein. Convolutional neural networks have recently shown their superior- ity for this problem, bringing increased model expressiveness while remaining parameter ef- ficient. Despite the success, previous convolu- tion designs fail to model full interactions be- tween input entities and relations, which poten- tially limits the performance of link prediction. In this work we introduce ConvR, an adaptive convolutional network designed to maximize entity-relation interactions in a convolutional fashion. ConvR adaptively constructs convolu- tion filters from relation representations, and applies these filters across entity representa- tions to generate convolutional features. As such, ConvR enables rich interactions between entity and relation representations at diverse re- gions, and all the convolutional features gener- ated will be able to capture such interactions. We evaluate ConvR on multiple benchmark datasets. Experimental results show that: (1) ConvR performs substantially better than com- petitive baselines in almost all the metrics and on all the datasets; (2) Compared with state- of-the-art convolutional models, ConvR is not only more effective but also more efficient. It offers a 7% increase in MRR and a 6% in- crease in Hits@10, while saving 12% in pa- rameter storage. 1 Introduction Multi-relational data refers to directed graphs whose nodes correspond to entities and edges dif- ferent types of relations between entities. An edge of the form (subject, relation, object) indicates that there exists a specific relation between the subject and object entities. Learning with multi-relational data plays a pivotal role in many application do- mains, ranging from social networks or recom- mender systems to large-scale knowledge bases (KBs) (Bordes et al., 2013; Jenatton et al., 2012). This work focuses on modeling multi-relational data from KBs, with the aim of predicting miss- ing facts on KBs, a challenging task known as link prediction in statistical relational learning (SRL) (Getoor and Taskar, 2007). Various SRL techniques (Nickel et al., 2016a) have been proposed for this task, among which vec- tor space embedding models (Wang et al., 2017) are gaining increasing attention due to their su- perior performance and potential scalability. The key idea there is to learn and operate on latent fea- tures (embeddings) of entities and relations, so as to uncover non-trivial connectivity patterns in multi- relational data. Previous works of this kind tend to adopt shallow, simple models to extract latent features, e.g., the translation based models (Bordes et al., 2013; Wang et al., 2014) or the bilinear mod- els and their variants (Jenatton et al., 2012; Yang et al., 2015; Trouillon et al., 2016). Using these simple models allows one to easily handle large- scale KBs, but usually at the cost of learning …

Content truncated to 500 words