Inducing and Using Alignments for Transition-based AMR Parsing

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1086 - 1098 July 10-15, 2022 ©2022 Association for Computational Linguistics Inducing and Using Alignments for Transition-based AMR Parsing Andrew Drozdov† Jiawei Zhou‡ Radu Florian� Andrew McCallum† Tahira Naseem� Yoon Kim� Ramon Fernandez Astudillo� †UMass Amherst CICS ‡Harvard University �MIT CSAIL �IBM Research †{adrozdov,mccallum}@cs.umass.edu ‡jzhou02@g.harvard.edu �yoonkim@mit.edu �{raduf,tnaseem}@us.ibm.com �ramon.astudillo@ibm.com Abstract Transition-based parsers for Abstract Mean- ing Representation (AMR) rely on node-to- word alignments. These alignments are learned separately from parser training and require a complex pipeline of rule-based components, pre-processing, and post-processing to satisfy domain-specific constraints. Parsers also train on a point-estimate of the alignment pipeline, neglecting the uncertainty due to the inherent ambiguity of alignment. In this work we ex- plore two avenues for overcoming these limi- tations. First, we propose a neural aligner for AMR that learns node-to-word alignments with- out relying on complex pipelines. We subse- quently explore a tighter integration of aligner and parser training by considering a distribu- tion over oracle action sequences arising from aligner uncertainty. Empirical results show this approach leads to more accurate alignments and generalization better from the AMR2.0 to AMR3.0 corpora. We attain a new state-of- the art for gold-only trained models, matching silver-trained performance without the need for beam search on AMR3.0. 1 Introduction Abstract Meaning Representation (AMR) was in- troduced as an effort to unify various semantic tasks (entity-typing, co-reference, relation extrac- tion, and so on; Banarescu et al., 2013). Of exist- ing approaches for AMR parsing, transition-based parsing is particularly notable because it is high performing but still relies on node-to-word align- ments as a core pre-processing step (Ballesteros and Al-Onaizan, 2017; Liu et al., 2018; Naseem et al., 2019; Fernandez Astudillo et al., 2020; Zhou et al., 2021a,b, inter alia). These alignments are not in the training data and must be learned sepa- rately via a complex pipeline of rule-based sys- tems, pre-processing (e.g., lemmatization), and post-processing to satisfy domain-specific con- straints. Such pipelines can fail to generalize well, propagating errors into training that reduce AMR performance in new domains (e.g., AMR3.0). This work studies how we can probabilistically induce and use alignments for transition-based AMR pars- ing in a domain-agnostic manner, ultimately replac- ing the existing heuristics-based pipeline. To induce alignments, we propose a neural aligner which uses hard attention within a sequence- to-sequence model to learn latent alignments (Wu et al., 2018; Deng et al., 2018; Shankar et al., 2018). While straightforward, this neural parameterization makes it possible to easily incorporate pretrained features such as character-aware word embeddings from ELMo (Peters et al., 2018) and also relax some of the strong independence assumptions in classic count-based aligners such as IBM Model 1 (Brown et al., 1993). We find that the neural aligner meaningfully improves upon various baselines, in- cluding the existing domain-specific approach. To use the neural aligner’s posterior distribu- tion over alignments, we explore several methods. Our first approach simply uses the MAP alignment …

Content truncated to 500 words