Compositional Generalization for Neural Semantic Parsing via Span-level Supervised Attention

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2810–2823 June 6–11, 2021. ©2021 Association for Computational Linguistics 2810 Compositional Generalization for Neural Semantic Parsing via Span-level Supervised Attention Pengcheng Yin♠∗, Hao Fang♣, Graham Neubig♠, Adam Pauls♣, Emmanouil Antonios Platanios♣, Yu Su♣, Sam Thomson♣, Jacob Andreas♣ ♠Carnegie Mellon University ♣Microsoft Semantic Machines {pcyin,gneubig}@cs.cmu.edu {hao.fang,adam.pauls,anthony.platanios, yusu2,samuel.thomson,jacob.andreas}@microsoft.com Abstract We describe a span-level supervised attention loss that improves compositional generaliza- tion in semantic parsers. Our approach builds on existing losses that encourage attention maps in neural sequence-to-sequence models to imitate the output of classical word align- ment algorithms. Where past work has used word-level alignments, we focus on spans; bor- rowing ideas from phrase-based machine trans- lation, we align subtrees in semantic parses to spans of input sentences, and encourage neural attention mechanisms to mimic these alignments. This method improves the per- formance of transformers, RNNs, and struc- tured decoders on three benchmarks of com- positional generalization. 1 Introduction Semantic parsers translate natural language ut- terances (e.g., Schedule a meeting with Jean) into executable programs (e.g., CreateEvent( attendees=Jean)), and play a crucial role in ap- plications such as question answering systems and conversational agents (Liang, 2016; Gupta et al., 2018; Wen et al., 2017). As in many language understanding problems, a central challenge in se- mantic parsing is compositional generalization (Finegan-Dollak et al., 2018; Keysers et al., 2020). Consider a personal digital assistant for which de- velopers have assembled separate collections of annotated utterances for user requests involving their calendars (e.g., Schedule a meeting with Jean) and their contact books (e.g., Who is Jean’s man- ager?). An effective model should learn from this data how to additionally handle requests like Sched- ule a meeting with Jean’s manager, composing skills from the calendar and contacts domains, with little or no supervision for such combinations. Neural sequence-to-sequence models, which pro- vide the foundation for state-of-the-art semantic ∗ This work was mostly done during an internship at Microsoft Semantic Machines. parsers (Dong and Lapata, 2016; Yin and Neubig, 2017), tend to perform poorly at out-of-distribution generalization of this kind (Lake and Baroni, 2018; Furrer et al., 2020; Suhr et al., 2020). Methods have been proposed to bridge the generalization gap using meta-learning (Lake, 2019; Wang et al., 2020) or specialized model architectures (Russin et al., 2019; Li et al., 2019; Liu et al., 2020; Chen et al., 2020). These have registered impressive per- formance on small synthetic benchmark datasets, but it has proven difficult to effectively combine them with large-scale pre-training (Lewis et al., 2020; Raffel et al., 2020) and natural data (Furrer et al., 2020). In contrast to this extensive literature on data transformations and model architectures, the de- sign of loss functions to encourage compositional generalization has been under-explored. This pa- per investigates attention supervision losses that encourage attention matrices in neural sequence models to resemble the output of word alignment algorithms (Liu et al. (2016); Mi et al. (2016); Arthur et al. …

Content truncated to 500 words