Joint Language Semantic and Structure Embedding for Knowledge Graph Completion

Proceedings of the 29th International Conference on Computational Linguistics, pages 1965–1978 October 12–17, 2022. 1965 Joint Language Semantic and Structure Embedding for Knowledge Graph Completion Jianhao Shen1, Chenguang Wang2∗, Linyuan Gong3, Dawn Song3 1Peking University, 2Washington University in St. Louis, 3UC Berkeley jhshen@pku.edu.cn, chenguangwang@wustl.edu, {gly,dawnsong}@berkeley.edu Abstract The task of completing knowledge triplets has broad downstream applications. Both struc- tural and semantic information plays an im- portant role in knowledge graph completion. Unlike previous approaches that rely on either the structures or semantics of the knowledge graphs, we propose to jointly embed the se- mantics in the natural language description of the knowledge triplets with their structure in- formation. Our method embeds knowledge graphs for the completion task via fine-tuning pre-trained language models with respect to a probabilistic structured loss, where the forward pass of the language models captures seman- tics and the loss reconstructs structures. Our extensive experiments on a variety of knowl- edge graph benchmarks have demonstrated the state-of-the-art performance of our method. We also show that our method can significantly improve the performance in a low-resource regime, thanks to the better use of seman- tics. The code and datasets are available at https://github.com/pkusjh/LASS. 1 Introduction Knowledge graphs (KG), such as Wikidata and Freebase (Bollacker et al., 2008), consist of factual triplets. KGs have been useful resources for both humans and machines. A triplet in the form of (head entity, relation, tail entity), where the relation involves both head and tail entities, has been used in a great variety of applications, such as question answering (Guu et al.; Hao et al., 2017) and web search (Xiong et al., 2017). Incompleteness has been a longstanding issue in KGs (Carlson et al., 2010), impeding their wider adoption in real-world applications. KG completion aims to predict a missing entity or relation of a factual triplet. Structural patterns in the existing triplets are useful to predict the missing elements (Bordes et al., 2013; Sun et al., 2019). For ∗Corresponding author example, a composition pattern can be learned to predict the relation grandmother_Of based on two consecutive mother_Of relations. Besides the struc- ture information, semantic relatedness between en- tities and relations is also critical to infer entities or relations with similar meanings (An et al., 2018; Yao et al., 2019; Wang et al., 2021). For example, if a relationship CEO_Of holds between two enti- ties, the relation employee_Of also holds. There are two kinds of KG completion approaches that fall into different learning paradigms. First, the structure-based approaches treat entities and rela- tions as nodes and edges, and use graph embedding methods to learn their representations. Second, the semantic-based approaches encode the text descrip- tion of entities and relations via language models. While both structures and semantics are vital to KG completion, it is non-trivial for existing methods to process both structural and semantic information. In this paper, we propose LASS, a joint language semantic and structure embedding for knowledge graph completion, which incorporates both seman- tics and structures in a KG triplet. LASS …

Content truncated to 500 words