Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4524–4535 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 4524 Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation Jiaqi Guo1∗, Zecheng Zhan2∗, Yan Gao3, Yan Xiao3, Jian-Guang Lou3 Ting Liu1, Dongmei Zhang3 1Xi’an Jiaotong University, Xi’an, China 2Beijing University of Posts and Telecommunications, Beijing, China 3Microsoft Research Asia, Beijing, China jasperguo2013@stu.xjtu.edu.cn,zhanzecheng@bupt.edu.cn {Yan.Gao,Yan.Xiao,jlou,dongmeiz}@microsoft.com tingliu@mail.xjtu.edu.cn Abstract We present a neural approach called IRNet for complex and cross-domain Text-to-SQL. IR- Net aims to address two challenges: 1) the mismatch between intents expressed in natu- ral language (NL) and the implementation de- tails in SQL; 2) the challenge in predicting columns caused by the large number of out- of-domain words. Instead of end-to-end syn- thesizing a SQL query, IRNet decomposes the synthesis process into three phases. In the first phase, IRNet performs a schema link- ing over a question and a database schema. Then, IRNet adopts a grammar-based neural model to synthesize a SemQL query which is an intermediate representation that we design to bridge NL and SQL. Finally, IRNet deter- ministically infers a SQL query from the syn- thesized SemQL query with domain knowl- edge. On the challenging Text-to-SQL bench- mark Spider, IRNet achieves 46.7% accuracy, obtaining 19.5% absolute improvement over previous state-of-the-art approaches. At the time of writing, IRNet achieves the first po- sition on the Spider leaderboard. 1 Introduction Recent years have seen a great deal of renewed interest in Text-to-SQL, i.e., synthesizing a SQL query from a question. Advanced neural ap- proaches synthesize SQL queries in an end-to-end manner and achieve more than 80% exact match- ing accuracy on public Text-to-SQL benchmarks (e.g., ATIS, GeoQuery and WikiSQL) (Krishna- murthy et al., 2017; Zhong et al., 2017; Xu et al., 2017; Yaghmazadeh et al., 2017; Yu et al., 2018a; Dong and Lapata, 2018; Wang et al., 2018; Hwang et al., 2019). However, Yu et al. (2018c) yields unsatisfactory performance of state-of-the-art ap- ∗ Equal Contributions. Work done during an internship at MSRA. NL: Show the names of students who have a grade higher than 5 and have at least 2 friends. SQL: SELECT T1.name FROM friend AS T1 JOIN highschooler AS T2 ON T1.student_id = T2.id WHERE T2.grade > 5 GROUP BY T1.student_id HAVING count(*) >= 2 Figure 1: An example from the Spider benchmark to illustrate the mismatch between the intent expressed in NL and the implementation details in SQL. The column ‘student id’ to be grouped by in the SQL query is not mentioned in the question. proaches on a newly released, cross-domain Text- to-SQL benchmark, Spider. The Spider benchmark brings new challenges that prove to be hard for existing approaches. Firstly, the SQL queries in the Spider contain nested queries and clauses like GROUPBY and HAVING, which are far more complicated than that in another well-studied cross-domain bench- mark, WikiSQL (Zhong et al., 2017). Considering the example in Figure 1, the column ‘student id’ to be grouped by in …

Content truncated to 500 words