Structural Neural Encoders for AMR-to-text Generation

Proceedings of NAACL-HLT 2019, pages 3649–3658 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 3649 Structural Neural Encoders for AMR-to-text Generation Marco Damonte Shay B. Cohen School of Informatics, University of Edinburgh 10 Crichton Street, Edinburgh EH8 9AB, UK m.damonte@sms.ed.ac.uk scohen@inf.ed.ac.uk Abstract AMR-to-text generation is a problem recently introduced to the NLP community, in which the goal is to generate sentences from Ab- stract Meaning Representation (AMR) graphs. Sequence-to-sequence models can be used to this end by converting the AMR graphs to strings. Approaching the problem while working directly with graphs requires the use of graph-to-sequence models that encode the AMR graph into a vector representation. Such encoding has been shown to be beneficial in the past, and unlike sequential encoding, it al- lows us to explicitly capture reentrant struc- tures in the AMR graphs. We investigate the extent to which reentrancies (nodes with mul- tiple parents) have an impact on AMR-to-text generation by comparing graph encoders to tree encoders, where reentrancies are not pre- served. We show that improvements in the treatment of reentrancies and long-range de- pendencies contribute to higher overall scores for graph encoders. Our best model achieves 24.40 BLEU on LDC2015E86, outperforming the state of the art by 1.1 points and 24.54 BLEU on LDC2017T10, outperforming the state of the art by 1.24 points. 1 Introduction Abstract Meaning Representation (AMR; Ba- narescu et al. 2013) is a semantic graph represen- tation that abstracts away from the syntactic real- ization of a sentence, where nodes in the graph represent concepts and edges represent seman- tic relations between them. AMRs are graphs, rather than trees, because co-references and con- trol structures result in nodes with multiple par- ents, called reentrancies. For instance, the AMR of Figure 1(a) contains a reentrancy between fin- ger and he, caused by the possessive pronoun his. AMR-to-text generation is the task of auto- matically generating natural language from AMR (a) eat-01 he pizza finger :arg0 :arg1 :instrument part-of eat-01 :arg0 he :arg1 pizza :instr. finger :part-of he (b) eat-01 :arg0 he :arg1 pizza :instr. finger :part-of he (c) eat-01 :arg0 he :arg1 pizza :instr. finger :part-of he (d) Figure 1: (a) AMR for the sentence He ate the pizza with his fingers and different input representations: (b) sequential; (c) tree-structured; (d) graph-structured. The nodes and edges in bold highlight a reentrancy. graphs. Attentive encoder/decoder architectures, com- monly used for Neural Machine Translation (NMT), have been explored for this task (Kon- stas et al., 2017; Song et al., 2018; Beck et al., 2018). In order to use sequence-to-sequence mod- els, Konstas et al. (2017) reduce the AMR graphs to sequences, while Song et al. (2018) and Beck et al. (2018) directly encode them as graphs. Graph encoding allows the model to explicitly encode reentrant structures present in the AMR graphs. While central to AMR, reentrancies are often hard to treat both in parsing and in genera- tion. Previous work either removed them from the graphs, hence obtaining sequential (Konstas …

Content truncated to 500 words