Re-examining the Role of Schema Linking in Text-to-SQL

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 6943–6954, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 6943 Re-examining the Role of Schema Linking in Text-to-SQL Wenqiang Lei∗† , Weixin Wang∗†, Zhixin Ma†, Tian Gan‡, Wei Lu§, Min-Yen Kan†, Tat-Seng Chua† †National University of Singapore ‡Shandong University §Singapore University of Technology and Design wenqianglei@gmail.com zhixinma97@gmail.comweixin.wang@comp.nus.edu.sg gantian@sdu.edu.cn luwei@sutd.edu.sg kanmy@comp.nus.edu.sg dcscts@nus.edu.sg Abstract In existing sophisticated text-to-SQL models, schema linking is often considered as a simple, minor component, belying its importance. By providing a schema linking corpus based on the Spider text-to-SQL dataset, we systemati- cally study the role of schema linking. We also build a simple BERT-based baseline, called Schema-Linking SQL (SLSQL) to perform a data-driven study. We find when schema linking is done well, SLSQL demonstrates good performance on Spider despite its struc- tural simplicity. Many remaining errors are attributable to corpus noise. This suggests schema linking is the crux for the current text- to-SQL task. Our analytic studies provide in- sights on the characteristics of schema linking for future developments of text-to-SQL tasks.1 1 Introduction Structured Query Language (SQL), while exact and powerful, suffers from a complex grammar pre- senting significant challenges for laymen to write queries. Automatically parsing natural language into SQL (text-to-SQL) thus has huge potential, as it would enable lay users to mine the world’s structured data using natural language queries. To achieve practical text-to-SQL workflow, a model needs to correlate natural language queries with the given database. Therefore, schema link- ing is considered helpful for text-to-SQL pars- ing (Guo et al., 2019; Bogin et al., 2019b; Dong et al., 2019; Wang et al., 2020). Here, schema linking means identifying references of columns, tables and condition values in natural language queries. For example, for the question “Find the names of schools that have a donation with amount * Equal contribution. 1Our code and annotation are available at https:// github.com/WING-NUS/slsql. above 8.5” (shown with relevant tables in Figure 1), “name” is a column reference to school.name, “donation” a table reference to endowment, and “8.5” and “sale” are value references, corresponding to the condition values in the SQL query. Existing solutions largely treat schema linking as a minor component implemented with simple string matching (Guo et al., 2019; Yu et al., 2018a; Lin et al., 2019) heuristics to support sophisticated text- to-SQL models. An exception is Dong et al. (2019), which framed schema linking as a task to be solved by sequential tagging. While they did show the importance of schema linking, how it contribute to text-to-SQL task performance remains unanswered as there is no annotated corpus to analyze. To address these shortcomings, we perform an in- depth study on the role of schema linking in text-to- SQL parsing. Intuitively, schema linking helps both cross-domain generalizability and complex SQL generation, which have been identified as the cur- rent bottlenecks of the text-to-SQL task (Finegan- Dollak et al., 2018; Yu et al., 2018c). By cross- domain generalizability, we refer to the proper separation of training …

Content truncated to 500 words