Span-based Semantic Parsing for Compositional Generalization

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 908–921 August 1–6, 2021. ©2021 Association for Computational Linguistics 908 Span-based Semantic Parsing for Compositional Generalization Jonathan Herzig1 Jonathan Berant1,2 1Blavatnik School of Computer Science, Tel-Aviv University 2Allen Institute for Artificial Intelligence {jonathan.herzig,joberant}@cs.tau.ac.il Abstract Despite the success of sequence-to-sequence (seq2seq) models in semantic parsing, recent work has shown that they fail in composi- tional generalization, i.e., the ability to gen- eralize to new structures built of components observed during training. In this work, we posit that a span-based parser should lead to better compositional generalization. we pro- pose SPANBASEDSP, a parser that predicts a span tree over an input utterance, explic- itly encoding how partial programs compose over spans in the input. SPANBASEDSP ex- tends Pasupat et al. (2019) to be comparable to seq2seq models by (i) training from pro- grams, without access to gold trees, treating trees as latent variables, (ii) parsing a class of non-projective trees through an extension to standard CKY. On GEOQUERY, SCAN and CLOSURE datasets, SPANBASEDSP per- forms similarly to strong seq2seq baselines on random splits, but dramatically improves per- formance compared to baselines on splits that require compositional generalization: from 61.0 → 88.9 average accuracy. 1 Introduction The most dominant approach in recent years for semantic parsing, the task of mapping a natural lan- guage utterance to an executable program, has been based on sequence-to-sequence (seq2seq) models (Jia and Liang, 2016; Dong and Lapata, 2016; Wang et al., 2020, inter alia). In these models, the output program is decoded step-by-step (au- toregressively), using an attention mechanism that softly ties output tokens to the utterance. Despite the success of seq2seq models, recently, Finegan-Dollak et al. (2018) and Keysers et al. (2020) and Herzig and Berant (2019) demonstrated that such models fail at compositional generaliza- tion, that is, they do not generalize to program structures that were not seen at training time. For example, a model that observes at training time the questions “What states border China?” and “What is the largest state?” fails to generalize to questions such as “What states border the largest state?”. This is manifested in large performance drops on data splits designed to measure composi- tional generalization (compositional splits), and is in contrast to the generalization abilities of humans (Fodor and Pylyshyn, 1988). In this work, we posit that the poor generaliza- tion of seq2seq models is due to fact that the input utterance and output program are only tied softly through attention. We revisit a more traditional approach for semantic parsing (Zelle and Mooney, 1996; Zettlemoyer and Collins, 2005; Liang et al., 2011), where partial programs are predicted over short spans in the utterance, and are composed to build the program for the entire utterance. Such explicit inductive bias for compositionality should encourage compositional generalization. Specifically, we propose to introduce such induc- tive bias via a span-based parser (Stern et al., 2017; Pasupat et al., 2019), equipped …

Content truncated to 500 words