Pushing the Limits of AMR Parsing with Self-Learning

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3208–3214 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 3208 Pushing the Limits of AMR Parsing with Self-Learning Young-Suk Lee∗, Ram´on Fernandez Astudillo∗, Tahira Naseem∗, Revanth Gangi Reddy∗†, Radu Florian, Salim Roukos IBM Research {ysuklee,tnaseem}@us.ibm.com ramon.astudillo@ibm.com Abstract Abstract Meaning Representation (AMR) pars- ing has experienced a notable growth in perfor- mance in the last two years, due both to the im- pact of transfer learning and the development of novel architectures specific to AMR. At the same time, self-learning techniques have helped push the performance boundaries of other natural language processing applications, such as machine translation or question an- swering. In this paper, we explore different ways in which trained models can be applied to improve AMR parsing performance, includ- ing generation of synthetic text and AMR an- notations as well as refinement of actions ora- cle. We show that, without any additional hu- man annotations, these techniques improve an already performant parser and achieve state-of- the-art results on AMR 1.0 and AMR 2.0. 1 Introduction Abstract Meaning Representation (AMR) are broad-coverage sentence-level semantic represen- tations expressing who does what to whom. Nodes in an AMR graph correspond to concepts such as entities or predicates and are not always directly related to words. Edges in AMR represent relations between concepts such as subject/object. AMR has experienced unprecedented perfor- mance improvements in the last two years, partly due to the rise of pre-trained transformer models (Radford et al., 2019; Devlin et al., 2019; Liu et al., 2019), but also due to AMR-specific architecture improvements. A non-exhaustive list includes la- tent node-word alignments through learned per- mutations (Lyu and Titov, 2018a), minimum risk training via REINFORCE (Naseem et al., 2019), a sequence-to-graph modeling of linearized trees with copy mechanisms and re-entrance features ∗ Equal contribution. † Work done during AI Residency at IBM Research. (Zhang et al., 2019a) and more recently a highly performant graph-sequence iterative refinement model (Cai and Lam, 2020) and a hard-attention transition-based parser (F. A. et al., 2020), both based on the Transformer architecture. Given the strong improvements in architectures for AMR, it becomes interesting to explore alter- native avenues to push performance even further. AMR annotations are relatively expensive to pro- duce and thus typical corpora have on the order of tens of thousands of sentences. In this work we explore the use self-learning techniques as a means to escape this limitation. We explore the use of a trained parser to itera- tively refine a rule-based AMR oracle (Ballesteros and Al-Onaizan, 2017; F. A. et al., 2020) to yield better action sequences. We also exploit the fact that a single AMR graph maps to multiple sen- tences in combination with AMR-to-text (Mager et al., 2020), to generate additional training sam- ples without using external data. Finally we revisit silver data training (Konstas et al., 2017a). These techniques reach 77.3 and 80.7 Smatch (Cai and Knight, 2013) on AMR1.0 and AMR2.0 respec- tively using only gold …

Content truncated to 500 words