Generalized Shortest-Paths Encoders for AMR-to-Text Generation

Proceedings of the 28th International Conference on Computational Linguistics, pages 2004–2013 Barcelona, Spain (Online), December 8-13, 2020 2004 Generalized Shortest-Paths Encoders for AMR-to-Text Generation Lisa Jin and Daniel Gildea Department of Computer Science University of Rochester Rochester, NY 14627 Abstract For text generation from semantic graphs, past neural models encoded input structure via gated convolutions along graph edges. Although these operations provide local context, the distance messages can travel is bounded by the number of encoder propagation steps. We adopt recent efforts of applying Transformer self-attention to graphs to allow global feature propagation. In- stead of feeding shortest paths to the vertex self-attention module, we train a model to learn them using generalized shortest-paths algorithms. This approach widens the receptive field of a graph encoder by exposing it to all possible graph paths. We explore how this path diversity affects per- formance across levels of AMR connectivity, demonstrating gains on AMRs of higher reentrancy counts and diameters. Analysis of generated sentences also supports high semantic coherence of our models for reentrant AMRs. Our best model achieves a 1.4 BLEU and 1.8 CHRF++ margin over a baseline that encodes only pairwise-unique shortest paths. 1 Introduction please-01 man arg0 arg1 (a) The man is pleased with himself. want-01 think-01 listen-01 girl boy arg1 arg1 arg0 arg0 arg1 arg0 (b) The boy wants the girl to think he is listening to her. Figure 1: Sample AMRs. Text generation from sequences manifests in tasks such as story generation, sum- marization, and question-answering. Generation from semantic structures includ- ing Abstract Meaning Representation (Banarescu et al., 2013) instead involves graph inputs. An AMR is a rooted, tree-like structure that captures propositional meaning through labeled vertices and edges called concepts and relations. In this setting a model must recover the original sentence annotated by the AMR, which lacks syntactic specification (e.g., tense and plurality). The model must grapple with the denser connectivity of semantic graphs in addition to ambiguity in pre- dicting the surface string. The arbitrary structure of edges in a semantic graph allows it to express meaning compactly, yet also presents a challenge in learning global representations. Given sequential inputs, a neural encoder can repeatedly apply recurrent or convolutional operations to encode full strings. This local function sharing does not transfer easily to the varied neighborhoods of graphs, however. Previous neural AMR-to-text models (Beck et al., 2018; Song et al., 2018) have favored graph neural networks (GNNs) that iteratively update each vertex’s state as a function of those in its first-order neighborhood. To capture paths within the AMR, these models include gated recurrent units within the GNN vertex update function. Since paths may contain word order information, encoding them faith- fully could help a model’s decoder flatten the input graph into a target sequence. Though a GNN applies to arbitrary graphs, its number of stacked layers controls the length of encoded paths. Tuning this parameter is non-trivial since it depends on the size distribution of training graphs. This work is licensed under a Creative Commons …

Content truncated to 500 words