Global Reasoning over Database Structures for Text-to-SQL Parsing

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 3659–3664, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 3659 Global Reasoning over Database Structures for Text-to-SQL Parsing Ben Bogin1 Matt Gardner2 Jonathan Berant1,2 1School of Computer Science, Tel-Aviv University 2Allen Institute for Artificial Intelligence ben.bogin@cs.tau.ac.il, mattg@allenai.org, joberant@cs.tau.ac.il Abstract State-of-the-art semantic parsers rely on auto- regressive decoding, emitting one symbol at a time. When tested against complex databases that are unobserved at training time (zero- shot), the parser often struggles to select the correct set of database constants in the new database, due to the local nature of decoding. In this work, we propose a semantic parser that globally reasons about the structure of the output query to make a more contextually- informed selection of database constants. We use message-passing through a graph neural network to softly select a subset of database constants for the output query, conditioned on the question. Moreover, we train a model to rank queries based on the global alignment of database constants to question words. We apply our techniques to the current state-of- the-art model for SPIDER, a zero-shot seman- tic parsing dataset with complex databases, in- creasing accuracy from 39.4% to 47.4%. 1 Introduction The goal of zero-shot semantic parsing (Krishna- murthy et al., 2017; Xu et al., 2017; Yu et al., 2018b,a; Herzig and Berant, 2018) is to map lan- guage utterances into executable programs in a new environment, or database (DB). The key difficulty in this setup is that the parser must map new lexi- cal items to DB constants that weren’t observed at training time. Existing semantic parsers handle this mostly through a local similarity function between words and DB constants, which considers each word and DB constant in isolation. This function is com- bined with an auto-regressive decoder, where the decoder chooses the DB constant that is most simi- lar to the words it is currently attending to. Thus, selecting DB constants is done one at a time rather than as a set, and informative global considerations are ignored. Figure 1: An example where choosing a DB constant based on local similarities is difficult, but the ambiguity can be resolved through global reasoning (see text). Consider the example in Figure 1, where a ques- tion is mapped to a SQL query over a complex DB. After decoding SELECT, the decoder must now choose a DB constant. Assuming its attention is focused on the word ‘name’ (highlighted), and given local similarities only, the choice between the lexically-related DB constants (singer.name and song.name) is ambiguous. However, if we globally reason over the DB constants and ques- tion, we can combine additional cues. First, a sub- sequent word ‘nation’ is similar to the DB column country which belongs to the table singer, thus selecting the column singer.name from the same table is more likely. Second, the next ap- pearance of the word ‘name’ is next to the …

Content truncated to 500 words