Unsupervised Graph-Text Mutual Conversion with a Unified Pretrained Language Model

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 5130–5144 July 9-14, 2023 ©2023 Association for Computational Linguistics Unsupervised Graph-Text Mutual Conversion with a Unified Pretrained Language Model Yi Xu1, Shuqian Sheng1, Jiexing Qi1, Luoyi Fu1∗, Zhouhan Lin1 Xinbing Wang1, Chenghu Zhou2 1Shanghai Jiao Tong University, Shanghai, China 2IGSNRR, Chinese Academy of Sciences, Beijing, China {yixu98, susisheng, qi_jiexing, yiluofu, xwang8}@sjtu.edu.cn lin.zhouhan@gmail.com Abstract Graph-to-text (G2T) generation and text-to- graph (T2G) triple extraction are two essen- tial tasks for knowledge graphs. Existing un- supervised approaches become suitable can- didates for jointly learning the two tasks due to their avoidance of using graph-text parallel data. However, they adopt multiple complex modules and still require entity information or relation type for training. To this end, we pro- pose INFINITY, a simple yet effective unsu- pervised method with a unified pretrained lan- guage model that does not introduce external annotation tools or additional parallel informa- tion. It achieves fully unsupervised graph-text mutual conversion for the first time. Specifi- cally, INFINITY treats both G2T and T2G as a bidirectional sequence generation task by fine- tuning only one pretrained seq2seq model. A novel back-translation-based framework is then designed to generate synthetic parallel data au- tomatically. Besides, we investigate the im- pact of graph linearization and introduce the structure-aware fine-tuning strategy to alleviate possible performance deterioration via retain- ing structural information in graph sequences. As a fully unsupervised framework, INFINITY is empirically verified to outperform state-of- the-art baselines for G2T and T2G tasks. Addi- tionally, we also devise a new training setting called cross learning for low-resource unsuper- vised information extraction. 1 Introduction Graph-to-text (G2T) generation and text-to-graph (T2G) triple extraction are two mutually inverse tasks that are crucial to the domain of knowledge graphs (KGs). G2T verbalizes the structural in- formation in KG with descriptive texts, which has attracted much attention to expand the application scope of KG, such as KG-based dialogue and Q&A system (Ji et al., 2022). As a primary task of infor- mation extraction, T2G aims to extract triples from ∗ Luoyi Fu is the corresponding author. Alan Bean Test Pilot Apollo 12 NASA occupation crewMember operator Alfred Worden David Scott backupPilot commander Graph Text UT Austin, B.S. 1995 1963 selection almaMater Alan Bean graduated from UT Austin in 1955 with a Bachelor of Science degree. He was hired by NASA in 1963 and served as a test pilot. Apollo 12's backup pilot was Alfred Worden and was commanded by David Scott. Figure 1: A pair of knowledge subgraph and its corre- sponding text. text, the typical subtasks (He et al., 2020; Chen et al., 2022) of which include named entity recog- nition (NER) and relation extraction (RE). Figure 1 illustrates a training pair sample containing part of a knowledge graph and its corresponding text. G2T and T2G have been intensively studied re- spectively, mainly treated as two kinds of indepen- dent problems in a supervised way. Due to the suc- cess of pretrained language models …

Content truncated to 500 words