Low-Resource Compositional Semantic Parsing with Concept Pretraining

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1410–1419 May 2-6, 2023 ©2023 Association for Computational Linguistics Low-Resource Compositional Semantic Parsing with Concept Pretraining Subendhu Rongali∗ Amazon Alexa AI rongals@amazon.com Mukund Sridhar∗ Google smukund@google.com Haidar Khan Amazon Alexa AI khhaida@amazon.com Konstantine Arkoudas∗ Dyania Health konstantine@dyaniahealth.com Wael Hamza Amazon Alexa AI waelhamz@amazon.com Andrew McCallum UMass Amherst mccallum@iesl.cs.umass.edu Abstract Semantic parsing plays a key role in digital voice assistants such as Alexa, Siri, and Google Assistant by mapping natural language to struc- tured meaning representations. To extend the capabilities of a voice assistant for a new do- main, the underlying semantic parsing model needs to be retrained using thousands of anno- tated examples from the new domain, which is time-consuming and expensive. In this work, we present an architecture to perform such do- main adaptation automatically, with only a small amount of metadata about the new do- main and without any new training data (zero- shot) or with very few examples (few-shot). We use a base seq2seq (sequence-to-sequence) architecture and augment it with a concept en- coder that encodes intent and slot tags from the new domain. We also introduce a novel decoder-focused approach to pretrain seq2seq models to be concept aware using Wikidata. This pretraining helps our model learn impor- tant concepts and perform well in low-resource settings. We report few-shot and zero-shot re- sults for compositional semantic parsing on the TOPv2 dataset and show that our model outper- forms prior approaches in few-shot settings for the TOPv2 and SNIPS datasets. 1 Introduction Voice assistants such as Alexa, Siri, and Google Assistant often rely on semantic parsing to under- stand requests made by their users. The underly- ing semantic parsing model converts natural lan- guage user utterances into logical forms consisting of actions requested by the user (play music, check weather), called intents, and relevant entities in the request (which song? which location?), called slots. The model is built to process requests in a fixed set of domains, such as music, weather, shopping, and so on. With voice assistants increasingly pervading ∗ work done when SR was at UMass and MS and KA were at Amazon; it does not relate to their current positions. more aspects of daily life, systems need to be con- tinuously updated to comprehend new intents and slots across an ever-growing number of domains. Current semantic parsing models are trained on large amounts of annotated data from a predeter- mined set of domains. Extending these models to learn new intents or slots typically involves collect- ing and annotating large amounts of new data. This process is expensive and time-consuming. To com- bat this problem, researchers have proposed seman- tic parsing models that can be efficiently trained with fewer examples (few-shot) from new domains (Shrivastava et al., 2021; Mansimov and Zhang, 2021; Ghoshal et al., 2020; Shin et al., 2021; Desai et al., 2021; Rongali et al., 2022; Shrivastava et al., 2022). While these methods facilitate few-shot learning, they have …

Content truncated to 500 words