Towards Knowledge-Intensive Text-to-SQL Semantic Parsing with Formulaic Knowledge

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 5240 - 5253 December 7-11, 2022 ©2022 Association for Computational Linguistics Towards Knowledge-Intensive Text-to-SQL Semantic Parsing with Formulaic Knowledge Longxu Dou1∗, Yan Gao2, Xuqi Liu1, Mingyang Pan1, Dingzirui Wang1, Wanxiang Che1, Min-Yen Kan3, Dechen Zhan1, Jian-Guang Lou2 1 Harbin Institute of Technology 2 Microsoft Research Asia 3 National University of Singapore {lxdou, xqliu, mypan, dzrwang, car}@ir.hit.edu.cn, dechen@hit.edu.cn, {yan.gao, jlou}@microsoft.com, kanmy@comp.nus.edu.sg Abstract In this paper, we study the problem of knowledge-intensive text-to-SQL, in which domain knowledge is necessary to parse ex- pert questions into SQL queries over domain- specific tables. We formalize this scenario by building a new Chinese benchmark KNOWSQL consisting of domain-specific questions cov- ering various domains. We then address this problem by presenting formulaic knowledge, rather than by annotating additional data exam- ples. More concretely, we construct a formu- laic knowledge bank as a domain knowledge base and propose a framework (REGROUP) to leverage this formulaic knowledge during parsing. Experiments using REGROUP demon- strate a significant 28.2% improvement overall on KNOWSQL. 1 Introduction Text-to-SQL translates user queries into executable SQL, greatly facilitating interactions between users and relational databases. Along with the release of large-scale benchmarks (Zhong et al., 2017; Yu et al., 2018, 2019a,b) and developments in model design (Wang et al., 2020a; Cao et al., 2021), text- to-SQL works are now achieving promising results in both research and practical applications (Zeng et al., 2020). However, in the professional application of text- to-SQL, such as in the data analysis of financial reports, models require external knowledge to map the expert query with the domain-specific database. Take the financial query for example: What’s the EBIT1 of Walmart?, where the underlying database has component columns that can be used to calcu- late the EBIT. We treat this problem as knowledge- intensive text-to-SQL, where domain knowledge is highly necessary to parse expert questions over Asia.∗Contribution during the internship at Microsoft Research 1EBIT is Earnings Before Interest and Tax, and is calcu- lated as Revenue – Cost of Goods Sold – Operating Expenses. What‘s the balance of trade of BRIC countries? SELECT Ship_Out – Ship_In, Nation FROM Reports WHERE Nation in (‘Brazil’, ‘Russia’, ‘India’, ‘China’) Ground Parse Retrieval Balance of Trade = Exports - Imports BRIC countries: Country in {Brazil, Russia, India, China} Trade Surplus : Export > Import Formulaic Knowledge Bank Nation GDP Ship In Ship Out A 14.72 1550 2650 B 5.04 581 623 Figure 1: Harnessing REGROUP with formulaic knowl- edge for knowledge-intensive text-to-SQL with three steps: (1) Retrieval the formulaic knowledge; (2) Ground the concept of formulaic knowledge; (3) Parse the question. domain-specific tables. This problem prevents text- to-SQL techniques from being fielded in novel, professional applications to assist the experts in processing data. Traditional approaches would address this prob- lem by annotating specific question/SQL pairs on a target domain (Wang et al., 2015; Herzig and Berant, 2019). Then such mappings are induced during the training process. This approach does work but has the drawback …

Content truncated to 500 words