Ambiguous Learning from Retrieval: Towards Zero-shot Semantic Parsing

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 14081–14094 July 9-14, 2023 ©2023 Association for Computational Linguistics Ambiguous Learning from Retrieval: Towards Zero-shot Semantic Parsing Shan Wu1,3,∗, Chunlei Xin1,3,∗, Hongyu Lin1,†, Xianpei Han1,2, Cao Liu4, Jiansong Chen4, Fan Yang4, Guanglu Wan4 , Le Sun1,2,† 1Chinese Information Processing Laboratory 2State Key Laboratory of Computer Science Institute of Software, Chinese Academy of Sciences, Beijing, China 3University of Chinese Academy of Sciences, Beijing, China 4Meituan-Dianping Group {wushan2018,chunlei2021,hongyu2016,xianpei,sunle}@iscas.ac.cn, {liucao,chenjiansong,yangfan79,wanguanglu}@meituan.com Abstract Current neural semantic parsers mostly take supervised approaches, which require a consid- erable amount of expensive training data. As a result, minimizing supervision requirements has been one of the key challenges in semantic parsing. In this paper, we propose a Retrieval as Ambiguous Supervision framework, which can effectively collect high-coverage ambigu- ous supervisions (i.e., the parse candidates of an utterance) via a pre-trained language models- based retrieval system. Then, by assuming can- didates will contain the correct ones, the zero- shot task can be converted into an ambiguously supervised task. To improve the precision and coverage of such ambiguous supervision, we propose a confidence-driven self-training algo- rithm, in which a semantic parser is learned and exploited to disambiguate candidates iteratively. Experimental results show that our approach significantly outperforms the state-of-the-art zero-shot semantic parsing methods. 1 Introduction Semantic parsing aims to map natural language sentences into computer-understandable meaning representations(MRs), which has attracted substan- tial attention for many years (Wong and Mooney, 2007; Kate et al., 2005; Lu et al., 2008; Dong and Lapata, 2016). Nowadays, neural network methods have become the mainstream for semantic pars- ing. Since neural semantic parsers are limited to the patterns observed in the training data, a large number of annotated data is required. However, an- notating utterances with detailed, correct meaning representations is a difficult and time-consuming task, which relies on expert knowledge about MRs. Recent studies in semantic parsing try to employ pre-trained language models (PLMs) to alleviate the problem of data insufficiency. Shin et al. (2021); *Equally Contribution. †Corresponding Authors. Figure 1: The top-k accuracies of the retrieved MRs by PLMs-based retriever on the eight domains in OVERNIGHT. We can see that the retrieved results have high top-k accuracy but low precision. Wu et al. (2021); Schucher et al. (2022) reformu- late semantic parsing as constrained paraphrasing generation, where paraphrasing generation is mod- eled by PLMs. To eliminate the need for human- annotated data, Xu et al. (2020) employ PLMs to paraphrase repeatedly and obtain millions of data. However, these methods still rely on lots of detailed annotated data or heavy data synthesis. In this paper, we propose a Retrieval as Ambigu- ous Supervision (RaAS) framework for zero-shot semantic parsing, which is simple and effective. In the RaAS framework, we make full use of a PLMs- based retriever to return high-coverage candidates, and then convert zero-shot semantic parsing into ambiguously supervised semantic parsing1. As pre- vious work found, sentence similarity and PLMs can provide effective candidates: Herzig and Be- …

Content truncated to 500 words