Unsupervised Dual Paraphrasing for Two-stage Semantic Parsing

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6806–6817 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 6806 Unsupervised Dual Paraphrasing for Two-stage Semantic Parsing Ruisheng Cao Su Zhu Chenyu Yang Chen Liu Rao Ma Yanbin Zhao Lu Chen Kai Yu∗ MoE Key Lab of Artificial Intelligence SpeechLab, Department of Computer Science and Engineering AI Institute, Shanghai Jiao Tong University, China {211314,paul2204,yangcy,chris-chen,rm1031,zhaoyb}@sjtu.edu.cn {chenlusz,kai.yu}@sjtu.edu.cn Abstract One daunting problem for semantic parsing is the scarcity of annotation. Aiming to re- duce nontrivial human labor, we propose a two-stage semantic parsing framework, where the first stage utilizes an unsupervised para- phrase model to convert an unlabeled natu- ral language utterance into the canonical ut- terance. The downstream naive semantic parser accepts the intermediate output and re- turns the target logical form. Furthermore, the entire training process is split into two phases: pre-training and cycle learning. Three tailored self-supervised tasks are introduced throughout training to activate the unsuper- vised paraphrase model. Experimental re- sults on benchmarks OVERNIGHT and GE- OGRANNO demonstrate that our framework is effective and compatible with supervised train- ing. 1 Introduction Semantic parsing is the task of converting natu- ral language utterances into structured meaning representations, typically logical forms (Zelle and Mooney, 1996; Wong and Mooney, 2007; Zettle- moyer and Collins, 2007; Lu et al., 2008). One prominent approach to build a semantic parser from scratch follows this procedure (Wang et al., 2015): a). (canonical utterance, logical form) pairs are automatically generated according to a domain-general grammar and a domain- specific lexicon. b). Researchers use crowdsourcing to paraphrase those canonical utterances into natural lan- guage utterances (the upper part of Figure 1). c). A semantic parser is built upon collected (nat- ural language utterance, logical form) pairs. ∗The corresponding author is Kai Yu. Figure 1: Two-stage semantic parsing framework, which is composed of an unsupervised paraphrase model and a naive neural semantic parser. Canonical utterances are pseudo-language utter- ances automatically generated from grammar rules, which can be understandable to people, but do not sound natural. Though effective, the paraphras- ing paradigm suffers from two drawbacks: (1) de- pendency on nontrivial human labor and (2) low utilization of canonical utterances. Annotators may struggle to understand the exact meanings of canonical utterances. Some canonical utterances even incur ambiguity, which enhances the difficulty of annotation. Furthermore, Wang et al. (2015) and Herzig and Berant (2019) only exploit them during data collection. Once the se- mantic parsing dataset is constructed, canonical utterances are thrown away, which leads to insuf- ficient utilization. While Berant and Liang (2014) and Su and Yan (2017) have reported the effective- ness of leveraging them as intermediate outputs, they experiment in a completely supervised way, where the human annotation is indispensable. In this work, inspired by unsupervised neural ma- chine translation (Lample et al., 2017; Artetxe et al., 2017), we propose a two-stage semantic parsing framework. The first stage uses a paraphrase model to convert natural language utterances into corre- 6807 sponding canonical utterances. …

Content truncated to 500 words