Neural AMR: Sequence-to-Sequence Models for Parsing and Generation

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 146–157 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1014 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 146–157 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1014 Neural AMR: Sequence-to-Sequence Models for Parsing and Generation Ioannis Konstas† Srinivasan Iyer† Mark Yatskar† Yejin Choi† Luke Zettlemoyer†‡ †Paul G. Allen School of Computer Science & Engineering, Univ. of Washington, Seattle, WA {ikonstas,sviyer,my89,yejin,lsz}@cs.washington.edu ‡Allen Institute for Artificial Intelligence, Seattle, WA lukez@allenai.org Abstract Sequence-to-sequence models have shown strong performance across a broad range of applications. However, their applica- tion to parsing and generating text using Abstract Meaning Representation (AMR) has been limited, due to the relatively lim- ited amount of labeled data and the non- sequential nature of the AMR graphs. We present a novel training procedure that can lift this limitation using millions of unla- beled sentences and careful preprocessing of the AMR graphs. For AMR parsing, our model achieves competitive results of 62.1 SMATCH, the current best score reported without significant use of external seman- tic resources. For AMR generation, our model establishes a new state-of-the-art performance of BLEU 33.8. We present extensive ablative and qualitative analysis including strong evidence that sequence- based AMR models are robust against ordering variations of graph-to-sequence conversions. 1 Introduction Abstract Meaning Representation (AMR) is a se- mantic formalism to encode the meaning of natu- ral language text. As shown in Figure 1, AMR rep- resents the meaning using a directed graph while abstracting away the surface forms in text. AMR has been used as an intermediate meaning repre- sentation for several applications including ma- chine translation (MT) (Jones et al., 2012), sum- marization (Liu et al., 2015), sentence compres- sion (Takase et al., 2016), and event extraction (Huang et al., 2016). While AMR allows for rich semantic representation, annotating training data in AMR is expensive, which in turn limits the use Obama was elected and his voters celebrated Obama elect.01 celebrate.01 vote.01 and * op1 op2 ARG0 poss ARG0 person name name op1 person ARG0-of Figure 1: respondingAn example sentence and its cor- Abstract Meaning Representation (AMR). AMR encodes semantic dependencies be- tween entities mentioned in the sentence, such as “Obama” being the “arg0” of the verb “elected”. of neural network models (Misra and Artzi, 2016; Peng et al., 2017; Barzdins and Gosko, 2016). In this work, we present the first success- ful sequence-to-sequence (seq2seq) models that achieve strong results for both text-to-AMR pars- ing and AMR-to-text generation. Seq2seq models have been broadly successful in many other appli- cations (Wu et al., 2016; Bahdanau et al., 2015; Luong et al., 2015; Vinyals et al., 2015). How- ever, their application to AMR has been limited, in part because effective linearization (encoding graphs as linear sequences) and data sparsity were thought to pose significant challenges. We show that these challenges can be easily overcome, by demonstrating that seq2seq models …

Content truncated to 500 words