On the Potential of Lexico-logical Alignments for Semantic Parsing to SQL Queries

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1849–1864 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 1849 On the Potential of Lexico-logical Alignments for Semantic Parsing to SQL Queries Tianze Shi∗ Cornell University tianze@cs.cornell.edu Chen Zhao∗ University of Maryland chenz@cs.umd.edu Jordan Boyd-Graber University of Maryland jbg@umiacs.umd.edu Hal Daum´e III Microsoft Research & University of Maryland me@hal3.name Lillian Lee Cornell University llee@cs.cornell.edu Abstract Large-scale semantic parsing datasets anno- tated with logical forms have enabled major advances in supervised approaches. But can richer supervision help even more? To ex- plore the utility of fine-grained, lexical-level supervision, we introduce SQUALL, a dataset that enriches 11,276 WIKITABLEQUESTIONS English-language questions with manually cre- ated SQL equivalents plus alignments between SQL and question fragments. Our annotation enables new training possibilities for encoder- decoder models, including approaches from machine translation previously precluded by the absence of alignments. We propose and test two methods: (1) supervised attention; (2) adopting an auxiliary objective of disam- biguating references in the input queries to ta- ble columns. In 5-fold cross validation, these strategies improve over strong baselines by 4.4% execution accuracy. Oracle experiments suggest that annotated alignments can support further accuracy gains of up to 23.9%. 1 Introduction The availability of large-scale datasets pairing natu- ral utterances with logical forms (Dahl et al., 1994; Wang et al., 2015; Zhong et al., 2017; Yu et al., 2018, inter alia) has enabled significant progress on supervised approaches to semantic parsing (Jia and Liang, 2016; Xiao et al., 2016; Dong and Lapata, 2016, 2018, inter alia). However, the provision of logical forms alone does not indicate important fine- grained relationships between individual words or phrases and logical form tokens. This is unfortu- nate because researchers have in fact hypothesized that the lack of such alignment information ham- pers progress in semantic parsing (Zhang et al., 2019, pg. 80). ∗Equal contribution; listed in alphabetical order. Table: Athlete (c1) Total Time (c2) Total Rank (c3) … Stefan Shalamanov 1:52.37 23 … Borislav Dimitrachkov 1:50.81 19 … Petar Popangelov 1:46.34 16 … Question: Target Logical Form: Answer: Bulgaria at the 1988 Winter Olympics Petar Popangelov Table: City (c1) Population (c2) Area (km2) (c3) … Alessandria 94191 203.97 … Casale Monferrato 36039 86.32 … Novi Ligure 28581 54.22 … Tortona 27476 99.29 … Acqui Terme 20426 33.42 … Question: Target Logical Form: SELECT count(c1) FROM w WHERE c2_number>=25000 Answer: Province of Alessandria 4 ① ② ③ ④ ⑤ How many cities have at least 25,000 people? ① ② ① ⑤ ④ ③ ① ② ③ Who has the highest rank ? ① ② ③ ② SELECT c1 FROM w ORDER BY c3_number LIMIT 1 Figure 1: Two examples from SQUALL. The table-question-answer triplets come from WIKITABLE- QUESTIONS. We provide the logical forms as SQL plus alignments between question and logical form. In the bottom example, for instance, “the highest” ↔ ORDER BY and LIMIT 1, as indicated by both matching high- light color ( blue ) and circled-number labels ( 2 ). We …

Content truncated to 500 words