Rewarding Smatch: Transition-Based AMR Parsing with Reinforcement Learning

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4586–4592 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 4586 Rewarding Smatch: Transition-Based AMR Parsing with Reinforcement Learning Tahira Naseem♠ Abhishek Shah♦ Hui Wan♠ Radu Florian♠ Salim Roukos♠ Miguel Ballesteros♠ ♠IBM Research, Yorktown Heights, NY, USA ♦IBM Watson, New York, NY, USA tnaseem, hwan, raduf, roukos@us.ibm.com abhishek.shah1, miguel.ballesteros@ibm.com Abstract Our work involves enriching the Stack-LSTM transition-based AMR parser (Ballesteros and Al-Onaizan, 2017) by augmenting training with Policy Learning and rewarding the Smatch score of sampled graphs. In addition, we also combined several AMR-to-text align- ments with an attention mechanism and we supplemented the parser with pre-processed concept identification, named entities and con- textualized embeddings. We achieve a highly competitive performance that is comparable to the best published results. We show an in- depth study ablating each of the new compo- nents of the parser. 1 Introduction Abstract meaning representations (AMRs) (Ba- narescu et al., 2013) are rooted labeled di- rected acyclic graphs that represent a non inter- sentential abstraction of natural language with broad-coverage semantic representations. AMR parsing thus requires solving several natural lan- guage processing tasks; named entity recognition, word sense disambiguation and joint syntactic and semantic role labeling. AMR parsing has acquired a lot of attention (Wang et al., 2015a; Zhou et al., 2016; Wang et al., 2015b; Goodman et al., 2016; Guo and Lu, 2018; Lyu and Titov, 2018; Vilares and Gómez-Rodríguez, 2018; Zhang et al., 2019) in recent years. We build upon a transition-based parser (Balles- teros and Al-Onaizan, 2017) that uses Stack- LSTMs (Dyer et al., 2015). We augment train- ing with self-critical policy learning (Rennie et al., 2017) using sentence-level Smatch scores (Cai and Knight, 2013) as reward. This objective is partic- ularly well suited for AMR parsing, since it over- comes the issues arising from the lack of token- level AMR-to-text alignments. In addition, we perform several modifications which are inspired from neural machine translation (Bahdanau et al., 2014) and by the recent trends on contextualized representations (Peters et al., 2018; Devlin et al., 2018). Our contributions are: (1) combinations of dif- ferent alignment methods: There has been signif- icant research in that direction (Flanigan et al., 2014; Pourdamghani et al., 2014; Chen, 2015; Chu and Kurohashi, 2016; Chen and Palmer, 2017; Szubert et al., 2018; Liu et al., 2018). In this paper, we show that combination of different methods makes a positive impact. We also combine hard alignments with an attention mechanism (Bah- danau et al., 2014). (2) Preprocessing of named entities and concepts. (3) Incorporating contextu- alized vectors (with BERT) and compare their ef- fectiveness with detailed ablation experiments. (4) Employing policy gradient training algorithm that uses Smatch as reward. 2 Stack-LSTM AMR Parser We use the Stack-LSTM transition based AMR parser of Ballesteros and Al-Onaizan (2017) (henceforth, we refer to it as BO). BO follows the Stack-LSTM dependency parser by Dyer et al. (2015). This approach allows unbounded looka- head and makes …

Content truncated to 500 words