Factorising AMR generation through syntax

Proceedings of NAACL-HLT 2019, pages 2157–2163 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2157 Factorising AMR generation through syntax Kris Cao and Stephen Clark Computer Laboratory University of Cambridge United Kingdom {kc391,sc609}@cam.ac.uk Abstract Generating from Abstract Meaning Represen- tation (AMR) is an underspecified problem, as many syntactic decisions are not constrained by the semantic graph. To explicitly account for this underspecification, we break down generating from AMR into two steps: first generate a syntactic structure, and then gen- erate the surface form. We show that decom- posing the generation process this way leads to state-of-the-art single model performance generating from AMR without additional un- labelled data. We also demonstrate that we can generate meaning-preserving syntactic para- phrases of the same AMR graph, as judged by humans. 1 Introduction Abstract Meaning Representation (AMR) (Ba- narescu et al., 2013) is a semantic annotation framework which abstracts away from the surface form of text to capture the core ‘who did what to whom’ structure. As a result, generating from AMR is underspecified (see Figure 1 for an exam- ple). Single-step approaches to AMR generation (Flanigan et al., 2016; Konstas et al., 2017; Song et al., 2016, 2017) therefore have to decide the syntax and surface form of the AMR realisation in one go. We instead explicitly try and capture this syntactic variation and factor the generation process through a syntactic representation (Walker et al., 2001; Duˇsek and Jurcicek, 2016; Gardent and Perez-Beltrachini, 2017; Currey and Heafield, 2018). First, we generate a delexicalised constituency structure from the AMR graph using a syntax model. Then, we fill out the constituency structure with the semantic content in the AMR graph using a lexicalisation model to generate the final surface form. Breaking down the AMR generation pro- cess this way provides us with several advantages: we disentangle the variance caused by the choice of syntax from that caused by the choice of words. We can therefore realise the same AMR graph with a variety of syntactic structures by sampling from the syntax model, and deterministically de- coding using the lexicalisation model. We hypoth- esise that this generates better paraphrases of the reference realisation than sampling from a single- step model. We linearise both the AMR graphs (Konstas et al., 2017) and constituency trees (Vinyals et al., 2015b) to allow us to use sequence-to-sequence models (Sutskever et al., 2014; Bahdanau et al., 2015) for the syntax and lexicalisation models. Further, as the AMR dataset is relatively small, we have issues with data sparsity causing poor param- eter estimation for rarely seen words. We deal with this by anonymizing named entities, and including a copy mechanism (Vinyals et al., 2015a; See et al., 2017; Song et al., 2018) into our decoder, which allows open-vocabulary token generation. We show that factorising the generation process in this way leads to improvements in AMR genera- tion, setting a new state of the art for single-model AMR generation performance training only on la- belled data. We also …

Content truncated to 500 words