Few-shot Knowledge Graph-to-Text Generation with Pretrained Language Models

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1558–1568 August 1–6, 2021. ©2021 Association for Computational Linguistics 1558 Few-shot Knowledge Graph-to-Text Generation with Pretrained Language Models Junyi Li1,3, Tianyi Tang1, Wayne Xin Zhao1,3,5∗, Zhicheng Wei4, Nicholas Jing Yuan4 and Ji-Rong Wen1,2,3 1Gaoling School of Artificial Intelligence, Renmin University of China 2School of Information, Renmin University of China 3Beijing Key Laboratory of Big Data Management and Analysis Methods 4Huawei Cloud 5Beijing Academy of Artificial Intelligence, Beijing, 100084, China {lijunyi,jrwen,steven_tang}@ruc.edu.cn {batmanfly,nicholas.jing.yuan}@gmail.com weizhicheng1@huawei.com Abstract This paper studies how to automatically gen- erate a natural language text that describes the facts in knowledge graph (KG). Considering the few-shot setting, we leverage the excel- lent capacities of pretrained language models (PLMs) in language understanding and gener- ation. We make three major technical contri- butions, namely representation alignment for bridging the semantic gap between KG encod- ings and PLMs, relation-biased KG lineariza- tion for deriving better input representations, and multi-task learning for learning the cor- respondence between KG and text. Exten- sive experiments on three benchmark datasets have demonstrated the effectiveness of our model on KG-to-text generation task. In par- ticular, our model outperforms all compari- son methods on both fully-supervised and few- shot settings. Our code and datasets are avail- able at https://github.com/RUCAIBox/ Few-Shot-KG2Text. 1 Introduction Knowledge graphs (KGs), such as Wikidata and DBpedia, are essential for many natural language processing (NLP) applications (Ji et al., 2020). To understand the structured information in KG, the task of KG-to-text generation has been proposed to automatically generate a descriptive text for a given knowledge graph (Koncel-Kedziorski et al., 2019; Ribeiro et al., 2020a). Figure 1 illustrates a KG with the corresponding descriptive text, in which the nodes (e.g., Stan Lee and Iron Man) represent entities and the edges (e.g., creator and alias) de- scribe the relations between connected entities. In recent years, with the help of crowdsourcing platforms and information extraction (IE) systems, large-scale labelled pairs of KG and its descrip- tive text have been created, such as WikiBio (Le- bret et al., 2016) and WebNLG Challenge (Gardent ∗Corresponding author Jack Kirby creator Stan Lee creator Stan Lee founder The Avengers Thor founder Iron Man Tony Stark alias Iron Man is a fictional superhero who wears a suit of armor. He was created by writer Stan Lee, and designed by artists Jack Kirby . Iron Man's alter ego is Tony Stark. He has found the superhero team the Avengers alongside Thor. Entity mention KG Descriptive Text Figure 1: A knowledge graph (subgraph) with its de- scriptive text. The underlined words represent the con- text keywords about entities. et al., 2017). Based on these datasets, data-driven models have shown impressive capabilities to pro- duce informative and fluent text for a given KG (Lo- gan et al., 2019; Moryossef et al., 2019). However, due to the great expense in annotation process, it is not always feasible to generate large-scale la- belled datasets for a variety of domains in practice. Motivated by this, we propose to study the …

Content truncated to 500 words