Sequence-to-Sequence Knowledge Graph Completion and Question Answering

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 2814 - 2828 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Sequence-to-Sequence Knowledge Graph Completion and Question Answering Apoorv Saxena Indian Institute of Science Bangalore apoorvsaxena@iisc.ac.in Adrian Kochsiek University of Mannheim Germany adrian@informatik. uni-mannheim.de Rainer Gemulla University of Mannheim Germany rgemulla@uni-mannheim.de Abstract Knowledge graph embedding (KGE) models represent each entity and relation of a knowl- edge graph (KG) with low-dimensional em- bedding vectors. These methods have recently been applied to KG link prediction and ques- tion answering over incomplete KGs (KGQA). KGEs typically create an embedding for each entity in the graph, which results in large model sizes on real-world graphs with millions of en- tities. For downstream tasks these atomic en- tity representations often need to be integrated into a multi stage pipeline, limiting their utility. We show that an off-the-shelf encoder-decoder Transformer model can serve as a scalable and versatile KGE model obtaining state-of-the-art results for KG link prediction and incomplete KG question answering. We achieve this by posing KG link prediction as a sequence-to- sequence task and exchange the triple scoring approach taken by prior KGE methods with au- toregressive decoding. Such a simple but pow- erful method reduces the model size up to 98% compared to conventional KGE models while keeping inference time tractable. After finetun- ing this model on the task of KGQA over in- complete KGs, our approach outperforms base- lines on multiple large-scale datasets without extensive hyperparameter tuning.1 1 Introduction A knowledge graph (KG) is a multi-relational graph where the nodes are entities from the real world (e.g. Barack Obama, United States) and the named edges represent the relationships be- tween them (e.g. Barack Obama - born in - United States). KGs can be either domain-specific such as WikiMovies (Miller et al., 2016) or public, cross- domain KGs encoding common knowledge such as Wikidata and DBpedia (Heist et al., 2020). These graph-structured databases play an important role 1Resources are available at https://github.com/ apoorvumang/kgt5 in knowledge-intensive applications including web search, question answering and recommendation systems (Ji et al., 2020). Most real-world knowledge graphs are incom- plete. However, some missing facts can be in- ferred using existing facts in the KG (Bordes et al., 2013). This task termed knowledge graph com- pletion (KGC)2 has become a popular area of re- search in recent years (Wang et al., 2017) and is of- ten approached using knowledge graph embedding (KGE) models. KGE models represent each entity and relation of the KG by a dense vector embed- ding. Using these embeddings the model is trained to distinguish correct from incorrect facts. One of the main downstream applications of KGEs is question answering over incomplete KGs (KGQA) (Choudhary et al., 2021). Taking into account the large size of real world KGs (Wikidata contains ≈90M entities) and the applicability to downstream tasks, KGE models should fulfill the following desiderata: (i) scala- bility – i.e. have model size and inference time independent of the …

Content truncated to 500 words