Data Augmentation with Hierarchical SQL-to-Question Generation for Cross-domain Text-to-SQL Parsing

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8974–8983 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 8974 Data Augmentation with Hierarchical SQL-to-Question Generation for Cross-domain Text-to-SQL Parsing Kun Wu1∗, Lijie Wang2, Zhenghua Li1, Ao Zhang2, Xinyan Xiao2, Hua Wu2, Min Zhang1, Haifeng Wang2 1. Institute of Artificial Intelligence, School of Computer Science and Technology, Soochow University, Suzhou, China 2. Baidu Inc, Beijing, China kwu@stu.suda.edu.cn, {zhli13,minzhang}@suda.edu.cn {wanglijie,zhangao,xiaoxinyan,wu_hua,wanghaifeng}@baidu.com Abstract Data augmentation has attracted a lot of re- search attention in the deep learning era for its ability in alleviating data sparseness. The lack of labeled data for unseen evaluation databases is exactly the major challenge for cross-domain text-to-SQL parsing. Previous works either require human intervention to guarantee the quality of generated data, or fail to handle complex SQL queries. This pa- per presents a simple yet effective data aug- mentation framework. First, given a database, we automatically produce a large number of SQL queries based on an abstract syntax tree grammar. For better distribution matching, we require that at least 80% of SQL pat- terns in the training data are covered by gen- erated queries. Second, we propose a hi- erarchical SQL-to-question generation model to obtain high-quality natural language ques- tions, which is the major contribution of this work. Finally, we design a simple sampling strategy that can greatly improve training effi- ciency given large amounts of generated data. Experiments on three cross-domain datasets, i.e., WikiSQL and Spider in English, and DuSQL in Chinese, show that our proposed data augmentation framework can consistently improve performance over strong baselines, and the hierarchical generation component is the key for the improvement. 1 Introduction Given a natural language (NL) question and a rela- tional database (DB), the text-to-SQL parsing task aims to produce a legal and executable SQL query to get the correct answer (Date and Darwen, 1997), as depicted in Figure 1. A DB usually consists of multiple tables interconnected via foreign keys. Early research on text-to-SQL parsing mainly focuses on the in-domain setting (Li and Jagadish, 2014; Iyer et al., 2017; Yaghmazadeh et al., 2017), *Work done during an internship at Baidu Inc. Figure 1: An example of the text-to-SQL parsing task. where all question/SQL pairs of train/dev/test sets are generated against the same DB. In order to deal with the more realistic setting where DBs in the evaluation phase are unseen in the train- ing data, researchers propose several cross-domain datasets, such as WikiSQL (Zhong et al., 2017) and Spider (Yu et al., 2018b) in English, and DuSQL (Wang et al., 2020b) in Chinese. All three datasets adopt the DB-level data splitting, meaning that a DB and all its corresponding question/SQL pairs can appear in only one of the train/dev/test sets. Cross-domain text-to-SQL parsing has two ma- jor challenges. First, unseen DBs usually introduce new schemas, such as new table/column names and unknown semantics of inter-table relationships. Therefore, it is crucial for a parsing model to have strong generalization ability. The second challenge is that the …

Content truncated to 500 words