Confidence Driven Unsupervised Semantic Parsing

Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pages 1486–1495, Portland, Oregon, June 19-24, 2011. c⃝2011 Association for Computational Linguistics Confidence Driven Unsupervised Semantic Parsing Dan Goldwasser ∗ Roi Reichart † James Clarke ∗ Dan Roth ∗ ∗Department of Computer Science, University of Illinois at Urbana-Champaign {goldwas1,clarkeje,danr}@illinois.edu † Computer Science and Artificial Intelligence Laboratory, MIT roiri@csail.mit.edu Abstract Current approaches for semantic parsing take a supervised approach requiring a consider- able amount of training data which is expen- sive and difficult to obtain. This supervision bottleneck is one of the major difficulties in scaling up semantic parsing. We argue that a semantic parser can be trained effectively without annotated data, and in- troduce an unsupervised learning algorithm. The algorithm takes a self training approach driven by confidence estimation. Evaluated over Geoquery, a standard dataset for this task, our system achieved 66% accuracy, com- pared to 80% of its fully supervised counter- part, demonstrating the promise of unsuper- vised approaches for this task. 1 Introduction Semantic parsing, the ability to transform Natural Language (NL) input into a formal Meaning Repre- sentation (MR), is one of the longest standing goals of natural language processing. The importance of the problem stems from both theoretical and practi- cal reasons, as the ability to convert NL into a formal MR has countless applications. The term semantic parsing has been used ambigu- ously to refer to several semantic tasks (e.g., se- mantic role labeling). We follow the most common definition of this task: finding a mapping between NL input and its interpretation expressed in a well- defined formal MR language. Unlike shallow se- mantic analysis tasks, the output of a semantic parser is complete and unambiguous to the extent it can be understood or even executed by a computer system. Current approaches for this task take a data driven approach (Zettlemoyer and Collins, 2007; Wong and Mooney, 2007), in which the learning algorithm is given a set of NL sentences as input and their cor- responding MR, and learns a statistical semantic parser — a set of parameterized rules mapping lex- ical items and syntactic patterns to their MR. Given a sentence, these rules are applied recursively to de- rive the most probable interpretation. Since semantic interpretation is limited to the syn- tactic patterns observed in the training data, in or- der to work well these approaches require consider- able amounts of annotated data. Unfortunately an- notating sentences with their MR is a time consum- ing task which requires specialized domain knowl- edge and therefore minimizing the supervision ef- fort is one of the key challenges in scaling semantic parsers. In this work we present the first unsupervised approach for this task. Our model compensates for the lack of training data by employing a self training protocol based on identifying high confi- dence self labeled examples and using them to re- train the model. We base our approach on a sim- ple observation: semantic parsing is a difficult struc- tured prediction task, which …

Content truncated to 500 words