Lightweight, Dynamic Graph Convolutional Networks for AMR-to-Text Generation

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 2162–2172, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 2162 Lightweight, Dynamic Graph Convolutional Networks for AMR-to-Text Generation Yan Zhang∗1,2 Zhijiang Guo∗1 Zhiyang Teng3 Wei Lu1 Shay B. Cohen4 Zuozhu Liu5 Lidong Bing†2 1Singapore University of Technology and Design 2DAMO Academy, Alibaba Group, 3Westlake University 4University of Edinburgh, 5ZJU-UIUC Institute {yan zhang,zhijiang guo}@mymail.sutd.edu.sg, luwei@sutd.edu.sg tengzhiyang@westlake.edu.cn, scohen@inf.ed.ac.uk zuozhuliu@intl.zju.edu.cn, l.bing@alibaba-inc.com Abstract AMR-to-text generation is used to transduce Abstract Meaning Representation structures (AMR) into text. A key challenge in this task is to efficiently learn effective graph rep- resentations. Previously, Graph Convolution Networks (GCNs) were used to encode input AMRs, however, vanilla GCNs are not able to capture non-local information and additionally, they follow a local (first-order) information ag- gregation scheme. To account for these issues, larger and deeper GCN models are required to capture more complex interactions. In this paper, we introduce a dynamic fusion mecha- nism, proposing Lightweight Dynamic Graph Convolutional Networks (LDGCNs) that cap- ture richer non-local interactions by synthesiz- ing higher order information from the input graphs. We further develop two novel parame- ter saving strategies based on the group graph convolutions and weight tied convolutions to reduce memory usage and model complexity. With the help of these strategies, we are able to train a model with fewer parameters while maintaining the model capacity. Experiments demonstrate that LDGCNs outperform state- of-the-art models on two benchmark datasets for AMR-to-text generation with significantly fewer parameters. 1 Introduction Graph structures play a pivotal role in NLP because they are able to capture particularly rich structural information. For example, Figure 1 shows a di- rected, labeled Abstract Meaning Representation (AMR; Banarescu et al. 2013) graph, where each node denotes a semantic concept and each edge denotes a relation between such concepts. Within ∗ Equally Contributed. Work done while Yan Zhang was an intern at DAMO Academy, Alibaba Group and Zhijiang Guo was at the University of Edinburgh. †Corresponding author. join-01 (d) Structured SANs i game this kind join-01 (c) SANs i game this kind join-01 (b) LDGCNs i game this kind join-01 (a) Vanilla GCNs i game this kind ARG0 ARG1 mod mod mod mod mod mod mod mod ARG0 ARG0 ARG0 ARG1 ARG1 ARG1 Figure 1: The concept (join-01) in vanilla GCNs is that it only captures information from its immediate neigh- bors (first-order), while in LDGCNs it can integrate information from neighbors of different order (e.g., second-order and third-order). In SANs, the node col- lects information from all other nodes, while in struc- tured SANs it is aware of its connected nodes in the original graph. the realm of work on AMR, we focus in this pa- per on the problem of AMR-to-text generation, i.e. transducing AMR graphs into text that conveys the information in the AMR structure. A key challenge in this task is to efficiently learn useful represen- tations of the AMR graphs. Early efforts (Pour- damghani et al., 2016; Konstas et al., 2017) neglect a …

Content truncated to 500 words