Oxford at SemEval-2017 Task 9: Neural AMR Parsing with Pointer-Augmented Attention

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 914–919, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics Oxford at SemEval-2017 Task 9: Neural AMR Parsing with Pointer-Augmented Attention Jan Buys1 and Phil Blunsom1,2 1Department of Computer Science, University of Oxford 2DeepMind {jan.buys,phil.blunsom}@cs.ox.ac.uk Abstract We present an end-to-end neural encoder- decoder AMR parser that extends an attention-based model by predicting the alignment between graph nodes and sen- tence tokens explicitly with a pointer mechanism. Candidate lemmas are pre- dicted as a pre-processing step so that the lemmas of lexical concepts, as well as constant strings, are factored out of the graph linearization and recovered through the predicted alignments. The approach does not rely on syntactic parses or ex- tensive external resources. Our parser ob- tained 59% Smatch on the SemEval test set. 1 Introduction The task of parsing sentences to Abstract Meaning Representation (AMR) (Banarescu et al., 2013) has recently received increased attention. AMR represents sentence meaning with directed acyclic graphs (DAGs) with labelled nodes and edges. No assumptions are made about the relation between an AMR and the structure of the sentence it rep- resents: the representation is not assumed to have any relation to the sentence syntax, no alignments are given and no distinction is made between con- cepts that correspond directly to lexemes in the in- put sentences and those that don’t. This underspecification creates significant chal- lenges for training an end-to-end AMR parser, which are exacerbated by the relatively small sizes of available training sets. Consequently most AMR parsers are pipelines that make extensive use of additional resources. Neural encoder-decoders have previously been proposed for AMR pars- ing, but reported accuracies are well below the state-of-the-art (Barzdins and Gosko, 2016), even with sophisticated pre-processing and categoriza- tion (Peng et al., 2017). The end-to-end neural approach contrasts with approaches based on a pipeline of multiple LSTMs (Foland Jr and Mar- tin, 2016) or neural network classifiers inside a feature- and resource-rich parser (Damonte et al., 2017), which have performed competitively. Our approach addresses these challenges in two ways: This first is to utilize (noisy) align- ments, aligning each graph node to an input to- ken. The alignments are predicted explicitly by the neural decoder with a pointer network (Vinyals et al., 2015), in addition to a standard attention mechanism. Our second contribution is to in- troduce more structure in the AMR linearization by distinguishing between lexical and non-lexical concepts, noting that lexical concepts (excluding sense labels) can be predicted with high accuracy from their lemmas. The decoder predicts only delexicalized concepts, recovering the lexicaliza- tion through the lemmas corresponding to the pre- dicted alignments. Experiments show that our extensions increase parsing accuracy by a large margin over a standard attention-based model. 2 Graph Linearization and Lemmatization We start by discussing how to linearize AMR graphs to enable sequential prediction. AMR node labels are referred to as concepts and edge labels as relations. A special class of node modifiers, called constants, are …

Content truncated to 500 words