AMR Parsing with Action-Pointer Transformer

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5585–5598 June 6–11, 2021. ©2021 Association for Computational Linguistics 5585 AMR Parsing with Action-Pointer Transformer Jiawei Zhou� Tahira Naseem∗ Ramón Fernandez Astudillo∗ Radu Florian∗ �Harvard University ∗IBM Research �jzhou02@g.harvard.edu ∗{tnaseem,raduf}@us.ibm.com ∗ramon.astudillo@ibm.com Abstract Abstract Meaning Representation parsing is a sentence-to-graph prediction task where tar- get nodes are not explicitly aligned to sen- tence tokens. However, since graph nodes are semantically based on one or more sen- tence tokens, implicit alignments can be de- rived. Transition-based parsers operate over the sentence from left to right, capturing this inductive bias via alignments at the cost of limited expressiveness. In this work, we pro- pose a transition-based system that combines hard-attention over sentences with a target- side action pointer mechanism to decouple source tokens from node representations and address alignments. We model the transi- tions as well as the pointer mechanism through straightforward modifications within a single Transformer architecture. Parser state and graph structure information are efficiently en- coded using attention heads. We show that our action-pointer approach leads to increased expressiveness and attains large gains (+1.6 points) against the best transition-based AMR parser in very similar conditions. While using no graph re-categorization, our single model yields the second best SMATCH score on AMR 2.0 (81.8), which is further improved to 83.4 with silver data and ensemble decoding. 1 Introduction Abstract Meaning Representation (AMR) (Ba- narescu et al., 2013) is a sentence level semantic formalism encoding who does what to whom in the form of a rooted directed acyclic graph. Nodes rep- resent concepts such as entities or predicates which are not explicitly aligned to words, and edges repre- sent relations such as subject/object (see Figure 1). AMR parsing, the task of generating the graph from a sentence, is nowadays tackled with se- quence to sequence models parametrized with neu- ral networks. There are two broad categories of methods that are highly effective in recent years. Transition-based approaches predict a sequence of boy want-01 go-02 city name York New ARG0 ARG0 ARG4 name op1 op2 ARG1 Figure 1: AMR graph expressing the meaning of the sentence The boy wants to go to New York. actions given the sentence. These actions gener- ate the graph while processing tokens left-to-right through the sentence and store intermediate rep- resentations in memories such as stack and buffer (Wang et al., 2015; Damonte et al., 2016; Balles- teros and Al-Onaizan, 2017; Vilares and Gómez- Rodríguez, 2018; Naseem et al., 2019; Astudillo et al., 2020; Lee et al., 2020). General graph-based approaches, on the other hand, directly predict nodes and edges in sequential order from graph traversals such as breath first search or depth first search (Zhang et al., 2019a,b; Cai and Lam, 2019, 2020). While not modeling the local semantic cor- respondence between graph nodes and source to- kens, the approaches achieve strong results without restrictions of transition-based approaches, but of- ten require graph re-categorization, a form of graph …

Content truncated to 500 words