SPARQLing Database Queries from Intermediate Question Decompositions

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8984–8998 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 8984 SPARQLing Database Queries from Intermediate Question Decompositions Irina Saparina and Anton Osokin HSE University / Yandex / Moscow, Russia Abstract To translate natural language questions into executable database queries, most approaches rely on a fully annotated training set. Annotat- ing a large dataset with queries is difficult as it requires query-language expertise. We reduce this burden using grounded in databases inter- mediate question representations. These repre- sentations are simpler to collect and were orig- inally crowdsourced within the Break dataset (Wolfson et al., 2020). Our pipeline consists of two parts: a neural semantic parser that con- verts natural language questions into the in- termediate representations and a non-trainable transpiler to the SPARQL query language (a standard language for accessing knowledge graphs and semantic web). We chose SPARQL because its queries are structurally closer to our intermediate representations (compared to SQL). We observe that the execution accu- racy of queries constructed by our model on the challenging Spider dataset is comparable with the state-of-the-art text-to-SQL methods trained with annotated SQL queries. Our code and data are publicly available.1 1 Introduction The difficulty of collecting and annotating datasets for the task of translating a natural language ques- tion to an executable database query is a significant obstacle to the progress of the technology. The most popular multi-database text-to-SQL dataset, Spider (Yu et al., 2018), has 10K questions, which is smaller compared to question answering datasets of other types: the DROP dataset with text para- graphs has 97K questions (Dua et al., 2019) and the GQA dataset with images has 22M questions (Hud- son and Manning, 2019). The Spider dataset was created by 11 Yale students proficient in SQL, and it is difficult to scale such a process up. 1https://github.com/yandex-research/ sparqling-queries Recently, Wolfson et al. (2020) proposed the Question Decomposition Meaning Representation, QDMR, which is a way to decompose a question into a list of “atomic” steps representing an algo- rithm for answering the question. Importantly, they developed a crowdsourcing pipeline to annotate QDMRs and showed that it can be used at scale: they collected 83K QDMRs for questions (all in English) coming from different datasets (including Spider) and released them in the Break dataset. QDMRs resemble database queries but are not connected to any execution engine and cannot be run directly. Moreover, QDMRs were collected when looking only at questions and thus have no information about the database structure. Entities mentioned in QDMR steps usually have counter- parts in the corresponding database but do not have links to them (grounding). In this paper, we build a system for translating a natural language question first into QDMR and then into an executable query. We use modified QDMRs, where the entities described with text are replaced with their database groundings. Our sys- tem consists of two translators: a neural network for text-to-QDMR and a non-trainable QDMR-to- SPARQL transpiler. See Figure 1, …

Content truncated to 500 words