Retrofitting Distributional Embeddings to Knowledge Graphs with Functional Relations

Proceedings of the 27th International Conference on Computational Linguistics, pages 2423–2436 Santa Fe, New Mexico, USA, August 20-26, 2018. 2423 Retrofitting Distributional Embeddings to Knowledge Graphs with Functional Relations Benjamin J. Lengerich Carnegie Mellon University 5000 Forbes Avenue Pittsburgh, PA 15213 blengeri@cs.cmu.edu Andrew L. Maas and Christopher Potts Stanford University, Roam Analytics 195 East 4th Avenue San Mateo, CA 94401 {amaas, cgpotts}@roaminsight.com Abstract Knowledge graphs are a versatile framework to encode richly structured data relationships, but it can be challenging to combine these graphs with unstructured data. Methods for retrofitting pre-trained entity representations to the structure of a knowledge graph typically assume that entities are embedded in a connected space and that relations imply similarity. However, useful knowledge graphs often contain diverse entities and relations (with potentially disjoint underly- ing corpora) which do not accord with these assumptions. To overcome these limitations, we present Functional Retrofitting, a framework that generalizes current retrofitting methods by ex- plicitly modeling pairwise relations. Our framework can directly incorporate a variety of pairwise penalty functions previously developed for knowledge graph completion. Further, it allows users to encode, learn, and extract information about relation semantics. We present both linear and neural instantiations of the framework. Functional Retrofitting significantly outperforms existing retrofitting methods on complex knowledge graphs and loses no accuracy on simpler graphs (in which relations do imply similarity). Finally, we demonstrate the utility of the framework by predicting new drug–disease treatment pairs in a large, complex health knowledge graph. 1 Introduction Distributional representations of concepts are often easy to obtain from unstructured data sets, but they tend to provide only a blurry picture of the relationships that exist between concepts. In contrast, knowl- edge graphs directly encode this relational information, but it can be difficult to summarize the graph structure in a single representation for each entity. To combine the advantages of distributional and relational data, Faruqui et al. (2015) propose to retrofit embeddings learned from distributional data to the structure of a knowledge graph. Their method first learns entity representations based solely on distributional data and then applies a retrofitting step to update the representations based on the structure of a knowledge graph. This modular approach conve- niently separates the distributional data and entity representation learning from the knowledge graph and retrofitting model, allowing one to flexibly combine, reuse, and adapt existing representations to new tasks. However, a core assumption of Faruqui et al. (2015)’s retrofitting model is that connected entities should have similar embeddings. This assumption often fails to hold in large, complex knowledge graphs, for a variety of reasons. First, subgraphs of a knowledge graph often contain distinct classes of entities that are most naturally embedded in disconnected vector spaces. In the extreme case, the representations for these entities might derive from very different underlying data sets. For example, in a health knowledge graph, the subgraphs containing diseases and drugs should be allowed to form dis- joint vector spaces, and we might want to derive the initial representations from radically different data …

Content truncated to 500 words