Noise reduction and targeted exploration in imitation learning for Abstract Meaning Representation parsing

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 1–11, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Noise reduction and targeted exploration in imitation learning for Abstract Meaning Representation parsing James Goodman∗ Andreas Vlachos† Jason Naradowsky∗ ∗ Computer Science Department, University College London james@janigo.co.uk, jason.narad@gmail.com † Department of Computer Science, University of Sheffield a.vlachos@sheffield.ac.uk Abstract Semantic parsers map natural language statements into meaning representations, and must abstract over syntactic phenom- ena, resolve anaphora, and identify word senses to eliminate ambiguous interpre- tations. Abstract meaning representation (AMR) is a recent example of one such semantic formalism which, similar to a de- pendency parse, utilizes a graph to repre- sent relationships between concepts (Ba- narescu et al., 2013). As with dependency parsing, transition-based approaches are a common approach to this problem. How- ever, when trained in the traditional man- ner these systems are susceptible to the ac- cumulation of errors when they find un- desirable states during greedy decoding. Imitation learning algorithms have been shown to help these systems recover from such errors. To effectively use these meth- ods for AMR parsing we find it highly beneficial to introduce two novel exten- sions: noise reduction and targeted explo- ration. The former mitigates the noise in the feature representation, a result of the complexity of the task. The latter targets the exploration steps of imitation learning towards areas which are likely to provide the most information in the context of a large action-space. We achieve state-of- the art results, and improve upon standard transition-based parsing by 4.7 F1 points. 1 Introduction Meaning representation languages and systems have been devised for specific domains, such as ATIS for air-travel bookings (Dahl et al., 1994) and database queries (Zelle and Mooney, 1996; bolster against defenses center will attacks The NATO cyber ’s dobjprep nsubj aux pobj nn det possessive poss bolster-01 defend-01 center military attack-01 name cyber NATO ARG1 ARG0 ARG1 prep-against name mod op1 Figure 1: Dependency (left) and AMR graph (right) for: “The center will bolster NATO’s defenses against cyber-attacks.’ Liang et al., 2013). Such machine-interpretable representations enable many applications relying on natural language understanding. The ambi- tion of Abstract Meaning Representation (AMR) is that it is domain-independent and useful in a va- riety of applications (Banarescu et al., 2013). The first AMR parser by Flanigan et al. (2014) used graph-based inference to find a highest- scoring maximum spanning connected acyclic graph. Later work by Wang et al. (2015b) was in- spired by the similarity between the dependency parse of a sentence and its semantic AMR graph (Figure 1). Wang et al. (2015b) start from the de- pendency parse and learn a transition-based parser that converts it incrementally into an AMR graph using greedy decoding. An advantage of this ap- proach is that the initial stage of dependency pars- ing is well-studied and trained using larger corpora than that for which AMR annotations exist. Greedy decoding, where the parser builds the parse while maintaining only the best …

Content truncated to 500 words