Bridging the Gap between Synthetic and Natural Questions via Sentence Decomposition for Semantic Parsing

Bridging the Gap between Synthetic and Natural Questions via Sentence Decomposition for Semantic Parsing Yilin Niu1, Fei Huang1, Wei Liu2, Jianwei Cui2, Bin Wang2, Minlie Huang1∗ 1The CoAI Lab, DCST, Tsinghua University, Beijing, China; 1Institute for Artificial Intelligence, State Key Lab of Intelligent Technology and Systems, China; 1Beijing National Research Center for Information Science and Technology, China 2Xiaomi AI Lab, China niuyl14@tsinghua.org.cn f-huang18@mails.tsinghua.edu.cn {liuwei40,cuijianwei,wangbin11}@xiaomi.com aihuang@tsinghua.edu.cn Abstract Semantic parsing maps natural language ques- tions into logical forms, which can be executed against a knowledge base for answers. In real-world applications, the performance of a parser is often limited by the lack of training data. To facilitate zero-shot learn- ing, data synthesis has been widely studied to automatically generate paired questions and logical forms. However, data synthesis meth- ods can hardly cover the diverse structures in natural languages, leading to a large gap in sentence structure between synthetic and natural questions. In this paper, we propose a decomposition-based method to unify the sentence structures of questions, which ben- efits the generalization to natural questions. Experiments demonstrate that our method sig- nificantly improves the semantic parser trained on synthetic data (+7.9% on KQA and +8.9% on ComplexWebQuestions in terms of exact match accuracy). Extensive analysis demon- strates that our method can better generalize to natural questions with novel text expressions compared with baselines. Besides semantic parsing, our idea potentially benefits other se- mantic understanding tasks by mitigating the distracting structure features. To illustrate this, we extend our method to the task of sentence embedding learning, and observe substantial improvements on sentence retrieval (+13.1% for Hit@1). 1 Introduction Semantic parsing is a task of mapping natural lan- guage questions into logical forms, which serves as a backbone in knowledge base question answer- ing (Talmor and Berant, 2018; Gu et al., 2021; ∗Corresponding author. Cao et al., 2022) and task-oriented dialogue (Li et al., 2021a). Traditionally, learning a powerful semantic parser relies on large-scale annotated data, which is laborious to collect. To reduce the budget of annotation, some recent efforts have been dedicated into low-resource training methods, among which data synthesis is widely used. Data synthesis can efficiently generate large amounts of training data by means of templates and rules (Wang et al., 2015; Xu et al., 2020b), but the templated expressions of synthetic questions hinder the generalization to natural questions. The gap between synthetic and natural questions mainly derives from two kinds of textual fea- tures, phrase-level expression and sentence-level structure. The former refers to the myriad ways in which predicates, relations, and entities can be expressed (Berant and Liang, 2014), such as ‘‘the human whose date of birth is 2000’’ and ‘‘the person born in 2000’’. The latter usually manifests itself as the global rearrangement of content (see Figure 1 for an example). To allevi- ate the problem, paraphrasing is commonly used for increasing language diversity of synthetic data (Xu et al., 2020c; Weir et al., 2020). Although the paraphrasing methods successfully introduce diverse phrase-level expressions, they are still weak in generating sentences …

Content truncated to 500 words