LAGr: Label Aligned Graphs for Better Systematic Generalization in Semantic Parsing

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 3295 - 3308 May 22-27, 2022 c⃝2022 Association for Computational Linguistics LAGr: Label Aligned Graphs for Better Systematic Generalization in Semantic Parsing D´ora J´ambor∗ Mila - Quebec AI Institute McGill University dora.jambor@mail.mcgill.ca Dzmitry Bahdanau ServiceNow Research Mila - Quebec AI Institute McGill University Canada CIFAR AI Chair Abstract Semantic parsing is the task of producing structured meaning representations for natu- ral language sentences. Recent research has pointed out that the commonly-used sequence- to-sequence (seq2seq) semantic parsers strug- gle to generalize systematically, i.e. to han- dle examples that require recombining known knowledge in novel settings. In this work, we show that better systematic generalization can be achieved by producing the meaning representation directly as a graph and not as a sequence. To this end we propose LAGr (Label Aligned Graphs), a general framework to produce semantic parses by independently predicting node and edge labels for a com- plete multi-layer input-aligned graph. The strongly-supervised LAGr algorithm requires aligned graphs as inputs, whereas weakly- supervised LAGr infers alignments for orig- inally unaligned target graphs using approxi- mate maximum-a-posteriori inference. Exper- iments demonstrate that LAGr achieves signif- icant improvements in systematic generaliza- tion upon the baseline seq2seq parsers in both strongly- and weakly-supervised settings. 1 Introduction Recent research has shown that neural models struggle to systematically generalize to examples with unseen combinations of seen rules from the training set (Lake and Baroni, 2018; Finegan- Dollak et al., 2018; Hupkes et al., 2019). System- atic generalization is especially important for the task of semantic parsing, which requires models to translate natural language sentences to structured meaning representations (MRs), such as SPARQL database queries or lambda calculus logical forms. To generalize systematically in this task, the model must be capable of producing MRs for examples that feature new combinations of meaning construc- tion rules, such as the rule that maps a noun like ∗Corresponding author. Work partly done during an intern- ship at ServiceNow Research. Training example A hedgehog ate the cake ∗hedgehog(x1) ∧ cake(x4)∧ eat.agent(x2, x1) ∧ eat.theme(x2, x4) Generalization example The baby liked the hedgehog ∗baby(x1) ∧ hedgehog(x4)∧ like.agent(x2, x1) ∧ like.theme(x2, x4)) Figure 1: Examples from the training and the gener- alization sets of the COGS dataset (Kim and Linzen, 2020b). While “hedgehog” is only observed in the agent role during training, the generalization set fea- tures this word in the theme role. “hedgehog” in Figure 1 to its respective predicate hedgehog(.), and the rule that defines which se- mantic role with respect to the verb (e.g. agent or theme) the resulting predicate takes. Using syn- thetic (Bahdanau et al., 2019; Kim and Linzen, 2020a; Keysers et al., 2020) and natural bench- marks (Finegan-Dollak et al., 2018; Shaw et al., 2020), researchers have been studying systematic generalization of existing semantic parsing meth- ods as well as proposing new approaches such as using meta-learning (Conklin et al., 2021), pre- trained models (Furrer et al., 2020), or intermediate …

Content truncated to 500 words