Semantic Parsing Freebase: Towards Open-domain Semantic Parsing

Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 1: Proceedings of the Main Conference and the Shared Task, pages 328–338, Atlanta, Georgia, June 13-14, 2013. c⃝2013 Association for Computational Linguistics Semantic Parsing Freebase: Towards Open-domain Semantic Parsing Qingqing Cai Temple University Computer and Information Sciences qingqing.cai@temple.edu Alexander Yates Temple University Computer and Information Sciences yates@temple.edu Abstract Existing semantic parsing research has steadily improved accuracy on a few domains and their corresponding databases. This paper introduces FreeParser, a system that trains on one domain and one set of predicate and con- stant symbols, and then can parse sentences for any new domain, including sentences that refer to symbols never seen during training. FreeParser uses a domain-independent archi- tecture to automatically identify sentences relevant to each new database symbol, which it uses to supplement its manually-annotated training data from the training domain. In cross-domain experiments involving 23 domains, FreeParser can parse sentences for which it has seen comparable unannotated sentences with an F1 of 0.71. 1 Introduction Semantic parsing is the task of converting a sentence into a representation of its meaning, usually in a log- ical form grounded in the symbols of some fixed ontology or relational database (Zelle and Mooney, 1996; Zettlemoyer and Collins, 2005; Kate and Mooney, 2006). A growing body of research on semantic parsing has yielded consistent improve- ments in parsing accuracy. Yet existing semantic parsers have always been limited by the need for significant amounts of manually-annotated training data for each domain of discourse, or for each new database. As a result, current semantic parsers have been constrained to small domains, like answering geography questions. In an effort to break out of these narrowly- constrained domains, we investigate semantic parsers for Freebase, an online database of user- contributed facts divided into 86 domains, includ- ing everything from architecture to zoos. Freebase is much larger than standard benchmark databases for semantic parsing; for example, it contains 300 times as many relations, and 75,000 times as many in- stances, as the GeoQuery database. On average, the benchmark GeoQuery dataset has 125 training sen- tences per relation. An equivalent dataset for Free- base would require labeling close to 40,000 training sentences, an expensive undertaking. The size and diversity of data in Freebase forces us to consider a new task of open-domain seman- tic parsing. We introduce FreeParser, which trains on labeled examples from a select group of initial domains. It also uses the information in Freebase to automatically find unlabeled training sentences from Wikipedia for every Freebase relation. Using a self-supervised architecture, FreeParser automat- ically labels these sentences, and then trains a se- mantic parser for all of Freebase. The current re- striction to Wikipedia has a downside: 44% of the test questions in our dataset contained a word that never appeared in our set of automatically-collected sentences, suggesting that significant further gains could be had by scaling to a larger corpus. However, FreeParser is able to find correct parses for 70% of the questions from new …

Content truncated to 500 words