One Semantic Parser to Parse Them All: Sequence to Sequence Multi-Task Learning on Semantic Parsing Datasets

Proceedings of the 10th Conference on Lexical and Computational Semantics, pages 173–184 August 5–6, 2021, Bangkok, Thailand (online) ©2021 Association for Computational Linguistics 173 One Semantic Parser to Parse Them All: Sequence to Sequence Multi-Task Learning on Semantic Parsing Datasets Marco Damonte Emilio Monti Amazon Alexa AI {dammarco,monti}@amazon.com Abstract Semantic parsers map natural language utter- ances to meaning representations. The lack of a single standard for meaning representations led to the creation of a plethora of semantic parsing datasets. To unify different datasets and train a single model for them, we investi- gate the use of Multi-Task Learning (MTL) ar- chitectures. We experiment with five datasets (GEOQUERY, NLMAPS, TOP, OVERNIGHT, AMR). We find that an MTL architecture that shares the entire network across datasets yields competitive or better parsing accuracies than the single-task baselines, while reducing the total number of parameters by 68%. We fur- ther provide evidence that MTL has also bet- ter compositional generalization than single- task models. We also present a comparison of task sampling methods and propose a com- petitive alternative to widespread proportional sampling strategies. 1 Introduction Semantic parsing is the task of converting natural language into a meaning representation language (MRL). The commercial success of personal as- sistants, that are required to understand language, has contributed to a growing interest in semantic parsing. A typical use case for personal assistants is Question Answering (Q&A): the output of a se- mantic parser is a data structure that represents the underlying meaning of a given question. This data structure can be compiled into a query to retrieve the correct answer. The lack of a single standard for meaning representations resulted in the creation of a plethora of semantic parsing datasets, which differ in size, domain, style, complexity, and in the formalism used as an MRL. These datasets are expensive to create, as they normally require expert annotators. Consequently, the datasets are often limited in size. Multi-task Learning (MTL; Caruana 1997) refers to jointly learning several tasks while shar- ing parameters between them. In this paper, we use MTL to demonstrate that it is possible to unify these smaller datasets together to train a single model that can be used to parse sentences in any of the MRLs that appear in the data. We experiment with several Q&A semantic parsing dataset for English: GEOQUERY (Zelle and Mooney, 1996), NLMAPS V2 (Lawrence and Riezler, 2018b), TOP (Gupta et al., 2018), and OVERNIGHT (Wang et al., 2015b). In order to investigate the im- pact of less related tasks, we also experiment on a non-Q&A semantic parsing dataset, targeting a broader coverage meaning representation: AMR (Banarescu et al., 2013), which contains sentences from sources such as broadcasts, newswire, and discussion forums. Our baseline parsing architecture is a reimple- mentation of the sequence to sequence model by Rongali et al. (2020), which can be applied to any parsing task as long as the MRL can be expressed as a sequence. Inspired by Fan et al. (2017), we experimented with two MTL …

Content truncated to 500 words