RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQL

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3215 - 3229 December 7-11, 2022 ©2022 Association for Computational Linguistics RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQL Jiexing Qi1, Jingyao Tang1, Ziwei He1, Xiangpeng Wan2, Yu Cheng3 Chenghu Zhou4, Xinbing Wang1, Quanshi Zhang1, Zhouhan Lin1∗ 1Shanghai Jiao Tong University, Shanghai, China 2NetMind.AI and ProtagoLabs, Virginia, USA 3Microsoft Research, Redmond, Washington, USA 4IGSNRR, Chinese Academy of Sciences, Beijing, China {qi_jiexing, monstar, ziwei.he, zqs1022, xwang8}@sjtu.edu.cn lin.zhouhan@gmail.com Abstract Relational structures such as schema linking and schema encoding have been validated as a key component to qualitatively translating nat- ural language into SQL queries. However, in- troducing these structural relations comes with prices: they often result in a specialized model structure, which largely prohibits using large pretrained models in text-to-SQL. To address this problem, we propose RASAT: a Trans- former seq2seq architecture augmented with relation-aware self-attention that could lever- age a variety of relational structures while in- heriting the pretrained parameters from the T5 model effectively. Our model can incorporate almost all types of existing relations in the liter- ature, and in addition, we propose introducing co-reference relations for the multi-turn sce- nario. Experimental results on three widely used text-to-SQL datasets, covering both single- turn and multi-turn scenarios, have shown that RASAT could achieve state-of-the-art results across all three benchmarks (75.5% EX on Spi- der, 52.6% IEX on SParC, and 37.4% IEX on CoSQL). 1 1 Introduction Text-to-SQL is the task that aims at translating natural language questions into SQL queries. Since it could significantly break down barriers for non- expert users to interact with databases, it is among the most important semantic parsing tasks that are of practical importance (Kamath and Das, 2018; Deng et al., 2021). Various types of relations have been introduced for this task since Zhong et al. (2017) collected the first large-scale text-to-SQL dataset, which has resulted in significant boosts in the performance ∗ Zhouhan Lin is the corresponding author. 1Our implementation is available at https://github. com/LUMIA-group/rasat. through recent years. For example, Bogin et al. (2019b) introduced schema encoding to represent the schema structure of the database, and the result- ing augmented LSTM encoder-decoder architec- ture was able to generalize better towards unseen database schema. Lin et al. (2020a) introduced rela- tions between the entity mentioned in the question and the matched entries in the database to utilize database content effectively. Their BERT-based encoder is followed by an LSTM-based pointer net- work as the decoder, which generalizes better be- tween natural language variations and captures cor- responding schema columns more precisely. RAT- SQL (Wang et al., 2020a) introduced schema link- ing, which aligns mentions of entity names in the question to the corresponding schema columns or tables. Their augmented Transformer encoder is coupled with a specific tree-decoder. SADGA (Cai et al., 2021) introduced the dependency structure of the natural language question and designed a graph neural network-based encoder with a tree-decoder. On the other hand, a tree-decoder that can gener- ate grammatically …

Content truncated to 500 words