Semantic Parsing on Freebase from Question-Answer Pairs

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1533–1544, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Semantic Parsing on Freebase from Question-Answer Pairs Jonathan Berant Andrew Chou Roy Frostig Percy Liang Computer Science Department, Stanford University {joberant,akchou}@stanford.edu {rf,pliang}@cs.stanford.edu Abstract In this paper, we train a semantic parser that scales up to Freebase. Instead of relying on annotated logical forms, which is especially expensive to obtain at large scale, we learn from question-answer pairs. The main chal- lenge in this setting is narrowing down the huge number of possible logical predicates for a given question. We tackle this problem in two ways: First, we build a coarse mapping from phrases to predicates using a knowledge base and a large text corpus. Second, we use a bridging operation to generate additional predicates based on neighboring predicates. On the dataset of Cai and Yates (2013), despite not having annotated logical forms, our sys- tem outperforms their state-of-the-art parser. Additionally, we collected a more realistic and challenging dataset of question-answer pairs and improves over a natural baseline. 1 Introduction We focus on the problem of semantic parsing nat- ural language utterances into logical forms that can be executed to produce denotations. Traditional se- mantic parsers (Zelle and Mooney, 1996; Zettle- moyer and Collins, 2005; Wong and Mooney, 2007; Kwiatkowski et al., 2010) have two limitations: (i) they require annotated logical forms as supervision, and (ii) they operate in limited domains with a small number of logical predicates. Recent developments aim to lift these limitations, either by reducing the amount of supervision (Clarke et al., 2010; Liang et al., 2011; Goldwasser et al., 2011; Artzi and Zettle- moyer, 2011) or by increasing the number of logical Occidental College, Columbia University Execute on Database Type.University ⊓ Education.BarackObama Type.University Education BarackObama Which college did Obama go to ? alignment alignment bridging Figure 1: Our task is to map questions to answers via la- tent logical forms. To narrow down the space of logical predicates, we use a (i) coarse alignment based on Free- base and a text corpus and (ii) a bridging operation that generates predicates compatible with neighboring predi- cates. predicates (Cai and Yates, 2013). The goal of this paper is to do both: learn a semantic parser with- out annotated logical forms that scales to the large number of predicates on Freebase. At the lexical level, a major challenge in semantic parsing is mapping natural language phrases (e.g., “attend”) to logical predicates (e.g., Education). While limited-domain semantic parsers are able to learn the lexicon from per-example supervision (Kwiatkowski et al., 2011; Liang et al., 2011), at large scale they have inadequate coverage (Cai and Yates, 2013). Previous work on semantic parsing on Freebase uses a combination of manual rules (Yahya et al., 2012; Unger et al., 2012), distant supervision (Krishnamurthy and Mitchell, 2012), and schema 1533 matching (Cai and Yates, 2013). We use a large amount of web text and a knowledge base to build a …

Content truncated to 500 words