Graph-to-Text Generation with Dynamic Structure Pruning

Proceedings of the 29th International Conference on Computational Linguistics, pages 6115–6127 October 12–17, 2022. 6115 Graph-to-Text Generation with Dynamic Structure Pruning Liang Li1,2, Ruiying Geng3, Bowen Li3, Can Ma1∗, Yinliang Yue1, Binhua Li3 , Yongbin Li3∗ 1Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China 2School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China 3DAMO Academy, Alibaba Group {liliang, macan, yueyinliang}@iie.ac.cn {ruiying.gry, binhua.lbh, shuide.lyb}@alibaba-inc.com Abstract Most graph-to-text works are built on the encoder-decoder framework with cross- attention mechanism. Recent studies have shown that explicitly modeling the input graph structure can significantly improve the perfor- mance. However, the vanilla structural encoder cannot capture all specialized information in a single forward pass for all decoding steps, re- sulting in inaccurate semantic representations. Meanwhile, the input graph is flatted as an unordered sequence in the cross attention, ig- noring the original graph structure. As a re- sult, the obtained input graph context vector in the decoder may be flawed. To address these issues, we propose a Structure-Aware Cross- Attention (SACA) mechanism to re-encode the input graph representation conditioning on the newly generated context at each decoding step in a structure aware manner. We further adapt SACA and introduce its variant Dynamic Graph Pruning (DGP) mechanism to dynami- cally drop irrelevant nodes in the decoding pro- cess. We achieve new state-of-the-art results on two graph-to-text datasets, LDC2020T02 and ENT-DESC, with only minor increase on computational cost. 1 Introduction Data-to-text task aims to generate a natural lan- guage description from structural or semi-structural data, such as tables (Wiseman et al., 2017), Ab- stract Meaning Representation (AMR) graphs (Ba- narescu et al., 2013), and Knowledge Graphs (KG) (Cheng et al., 2020). It helps people get the key points of the input data and makes the stored in- formation accessible to a broader audience of end- users. There have been several practical application scenarios in this field, such as biography generation (Lebret et al., 2016), basketball news generation (Wiseman et al., 2017), and advertising text gen- eration (Shao et al., 2019). This paper focuses on ∗Corresponding authors: Can Ma, Yongbin Li Encoder Decoder Encoder Decoder Encoder Decoder Cross-Attention ×𝑁 (a) (b) Structure-Aware Cross-Attention 𝒉𝒕 ℎ" ℎ# ℎ"$% … GT context IG context (c) DGP structure-enhanced IG context Figure 1: (a) denotes an encoder-decoder framework with the cross-attention mechanism where IG and GT contexts denote the input graph and generated text graph contexts, respectively. (b) is an example of Structure- Aware Cross-Attention. The dotted lines in (c) denote the pruned edges and nodes. generation from graph structures in AMR and KG, referred to as graph-to-text. In recent years, encoder-decoder with the cross- attention mechanism has been the de facto frame- work for graph-to-text tasks (shown in Figure 1(a)). Given an input graph, the encoder first computes vector representations for the graph nodes. On the decoding side, Input Graph (IG) context vector is obtained via cross-attention based on the partially Generated Text (GT) at each time step, then the next target token is finally predicted. …

Content truncated to 500 words