Structural Information Preserving for Graph-to-Text Generation

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7987–7998 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 7987 Structural Information Preserving for Graph-to-Text Generation Linfeng Song1, Ante Wang2, Jinsong Su2⇤, Yue Zhang3, Kun Xu1, Yubin Ge4 and Dong Yu1 1Tencent AI Lab, Bellevue, WA, USA 2Xiamen University, Xiamen, China 3School of Engineering, Westlake University, China 4University of Illinois at Urbana-Champaign, USA Abstract The task of graph-to-text generation aims at producing sentences that preserve the mean- ing of input graphs. As a crucial defect, the current state-of-the-art models may mess up or even drop the core structural information of input graphs when generating outputs. We propose to tackle this problem by leveraging richer training signals that can guide our model for preserving input information. In particu- lar, we introduce two types of autoencoding losses, each individually focusing on different aspects (a.k.a. views) of input graphs. The losses are then back-propagated to better cali- brate our model via multi-task training. Exper- iments on two benchmarks for graph-to-text generation show the effectiveness of our ap- proach over a state-of-the-art baseline. Our code is available at http://github.com/ Soistesimmer/AMR-multiview. 1 Introduction Many text generation tasks take graph structures as their inputs, such as Abstract Meaning Representa- tion (AMR) (Banarescu et al., 2013), Knowledge Graph (KG) and database tables. For example, as shown in Figure 1(a), AMR-to-text generation is to generate a sentence that preserves the meaning of an input AMR graph, which is composed by a set of concepts (such as “boy” and “want-01”) and their relations (such as “:ARG0” and “:ARG1”). Similarly, as shown in Figure 1(b), KG-to-text gen- eration is to produce a sentence representing a KG, which contains worldwide factoid information of entities (such as “Australia” and “Above the Veil”) and their relations (such as “followedBy”). Recent efforts on graph-to-text generation tasks mainly focus on how to effectively represent input graphs, so that an attention mechanism can bet- ter transfer input knowledge to the decoder when ⇤Corresponding author followedBy (a) want-01 boy eat-01 ARG1 ARG0 ARG0 girl Above the Veil Aenir Into Battle precededBy Austrilia country (b) lunch ARG2 ARG1 beautiful mod Figure 1: (a) An AMR graph meaning “The boy wants the beautiful girl to eat lunch with him.”, and (b) A knowledge graph carrying the meaning “Above the Veil is an Australian novel and the sequel to Aenir. It was followed by Into the Battle.” generating sentences. Taking AMR-to-text genera- tion as an example, different graph neural networks (GNNs) (Beck et al., 2018; Song et al., 2018; Guo et al., 2019; Ribeiro et al., 2019) have been in- troduced to better represent input AMRs than a sequence-to-sequence model (Konstas et al., 2017), and later work (Zhu et al., 2019; Cai and Lam, 2019; Wang et al., 2020) showed that relation- aware Transformers can achieve even better results than GNNs. These advances for encoding have largely pushed the state-of-the-art performance. Existing models are optimized by maximizing the conditional word probabilities of a reference sentence, …

Content truncated to 500 words