Composing Relationships with Translations

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 286–290, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Composing Relationships with Translations Alberto Garc´ıa-Dur´an Sorbonne universit´es, UTC CNRS, Heudiasyc 7253 60203 Compi`egne, France agarciad@utc.fr Antoine Bordes Facebook AI Research 770 Broadway New York, NY. USA abordes@fb.com Nicolas Usunier Facebook AI Research 112, avenue de Wagram 75017 Paris, France usunier@fb.com Abstract Performing link prediction in Knowledge Bases (KBs) with embedding-based mod- els, like with the model TransE (Bordes et al., 2013) which represents relationships as translations in the embedding space, have shown promising results in recent years. Most of these works are focused on modeling single relationships and hence do not take full advantage of the graph structure of KBs. In this paper, we pro- pose an extension of TransE that learns to explicitly model composition of relation- ships via the addition of their correspond- ing translation vectors. We show empir- ically that this allows to improve perfor- mance for predicting single relationships as well as compositions of pairs of them. 1 Introduction Performing link prediction on multi-relational data is becoming essential in order to complete the huge amount of missing information of the knowl- edge bases. These knowledge can be formalized as directed multi-relation graphs, whose node cor- respond to entities connected with edges encod- ing various kind of relationships. We denote these connections via triples (head, label, tail). Link prediction consists in filling in incomplete triples like (head, label, ?) or (?, label, tail). In this context, embedding models (Wang et al., 2014; Lin et al., 2015; Jenatton et al., 2012; Socher et al., 2013) that attempt to learn low- dimensional vector or matrix representations of entities and relationships have shown promising performance in recent years. In particular, the ba- sic model TRANSE (Bordes et al., 2013) has been proved to be very powerful. This model treats each relationship as a translation vector operating on the embedding representing the entities. Hence, for a triple (head, label, tail), the vector embed- dings of head and tail are learned so that they are connected through a translation parameterized by the vector associated with label. Many extensions have been proposed to improve the representation power of TRANSE while still keeping its simplic- ity, by adding some projections steps before the translation (Wang et al., 2014; Lin et al., 2015). In this paper, we propose an extension of TRANSE 1 that focuses on improving its represen- tation of the underlying graph of multi-relational data by trying to learn compositions of relation- ships as sequences of translations in the embed- ding space. The idea is to train the embeddings by learning simple reasonings, such as the rela- tionship people/nationality should give a similar result as the composition people/city of birth and city/country. In our approach, called RTRANSE, the training set is augmented with relevant ex- amples of such compositions by performing con- strained walks in the knowledge graph, and train- ing so that sequences of translations lead to …

Content truncated to 500 words