Alignment-free Cross-lingual Semantic Role Labeling

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 422–434, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 422 AutoQA: From Databases To QA Semantic Parsers With Only Synthetic Training Data Silei Xu∗ Sina J. Semnani* Giovanni Campagna Monica S. Lam Computer Science Department Stanford University Stanford, CA, USA {silei,sinaj,gcampagn,lam}@cs.stanford.edu Abstract We propose AutoQA, a methodology and toolkit to generate semantic parsers that an- swer questions on databases, with no manual effort. Given a database schema and its data, AutoQA automatically generates a large set of high-quality questions for training that cov- ers different database operations. It uses au- tomatic paraphrasing combined with template- based parsing to find alternative expressions of an attribute in different parts of speech. It also uses a novel filtered auto-paraphraser to gener- ate correct paraphrases of entire sentences. We apply AutoQA to the Schema2QA dataset and obtain an average logical form accu- racy of 62.9% when tested on natural ques- tions, which is only 6.4% lower than a model trained with expert natural language anno- tations and paraphrase data collected from crowdworkers. To demonstrate the generality of AutoQA, we also apply it to the Overnight dataset. AutoQA achieves 69.8% answer ac- curacy, 16.4% higher than the state-of-the-art zero-shot models and only 5.2% lower than the same model trained with human data. 1 Introduction Semantic parsing is the task of mapping natural language sentences to executable logical forms. It has received significant attention in question an- swering systems for structured data (Wang et al., 2015; Zhong et al., 2017; Yu et al., 2018b; Xu et al., 2020). However, training a semantic parser with good accuracy requires a large amount of annotated data, which is expensive to acquire. The complex- ity of logical forms means annotating the data has to be done by an expert. This adds to the cost and hinders extending question answering to new databases and domains. ∗ Equal contribution Database schema and values Auto-Annotator Paraphraser POS-Based Annotation Extraction Template-based Data Synthesizer attribute annotations Auto-Paraphraser Paraphraser Semantic Parseri Paraphrase Filter TRAIN logical forms paraphrases paraphrases + original logical forms Figure 1: The architecture of the AutoQA toolkit. (a) The auto-annotator extracts annotations from para- phrases. (b) A template-based data synthesizer (Xu et al., 2020) generates data from the annotations to train a semantic parser. (c) An auto-paraphraser uses self-training to iteratively introduce more paraphrases to train the next version of the semantic parser. The red dotted lines show that generated paraphrases are fil- tered out unless the current semantic parser can trans- late them to the logical forms of the original sentences. To eliminate the need for annotating data with logical forms, SEMPRE (Wang et al., 2015) pro- posed the new methodology of first synthesizing questions on the database, then manually paraphras- ing them. Recently, the Schema2QA toolkit (Xu et al., 2020) demonstrated that it is possible to achieve high accuracy on realistic user inputs us- ing this methodology with a comprehensive set of generic, domain-independent question templates. However, this …

Content truncated to 500 words