NL2pSQL: Generating Pseudo-SQL Queries from Under-Specified Natural Language Questions

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 2603–2613, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 2603 NL2pSQL: Generating Pseudo-SQL Queries from Under-Specified Natural Language Questions Fuxiang Chen1,4, Seung-won Hwang2, Jaegul Choo3, Jung-Woo Ha1 and Sunghun Kim1,4 1Clova AI Research, NAVER 2Yonsei University 3Korea University 4Hong Kong University of Science and Technology fchenaa@cse.ust.hk , seungwonh@yonsei.ac.kr , jchoo@korea.ac.kr , {jungwoo.ha , sung.kim.n}@navercorp.com Abstract Generating SQL codes from natural language questions (NL2SQL) is an emerging research area. Existing studies have mainly focused on clear scenarios where specified informa- tion is fully given to generate a SQL query. However, in developer forums such as Stack Overflow,1 questions cover more diverse tasks including table manipulation or performance issues, where a table is not specified. The SQL query posted in Stack Overflow, Pseudo- SQL (pSQL), does not usually contain ta- ble schemas and is not necessarily executable, is sufficient to guide developers. Here we describe a new NL2pSQL task to generate pSQL codes from natural language questions on under-specified database issues, in short, NL2pSQL. In addition, we define two new metrics suitable for the proposed NL2pSQL task, Canonical-BLEU and SQL-BLEU, in- stead of the conventional BLEU. With a base- line model using sequence-to-sequence archi- tecture integrated with denoising autoencoder, we confirm the validity of our task. Experi- ments show that the proposed NL2pSQL ap- proach yields well-formed queries (up to 43% more than a standard Seq2Seq model). Our code and datasets are publicly available at http: //github.com/clovaai/nl2psql. 1 Introduction Converting natural-language questions to SQL codes (NL2pSQL) is an active area of research in natural language processing. However, earlier work (Hemphill et al., 1990; Brad et al., 2017; Zelle and Mooney, 1996; Tang and Mooney, 2000; Popescu et al., 2003; Lawrence and Riezler, 2016; Li and Jagadish, 2014; Roy et al., 2013; Yagh- mazadeh et al., 2017; Zhong et al., 2017; Yu et al., 2018c; Finegan-Dollak et al., 2018) mainly fo- cused on fully specified questions where the given 1https://www.stackoverflow.com question and the table can be uniquely answered by a generated SQL query. For example, a ques- tion such as “who is the manufacturer for the order year 1998?” can only be uniquely answered when the table that can answer the question is attached to the question. This requires the generated SQL query to be executable, in order to verify its cor- rectness. Meanwhile, real-life NL2pSQL questions from developer forums such as Stack Over- flow (SO) (StackOverflow, 2018) are mostly under-specified to generate an executable answer. Actual questions deal with more diverse issues, including table creation, backup, or performance, and often present with a less detailed context (such as missing table name or schema). For example, a SO question2 such as “How to reset AUTO INCREMENT in MySQL?” can only be answered by a non-executable template “ALTER TABLE [tablename] AUTO INCREMENT 1”, where human developers are expected to= read and then feed the missing context. We call …

Content truncated to 500 words