Information Extraction over Structured Data: Question Answering with Freebase

Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 956–966, Baltimore, Maryland, USA, June 23-25 2014. c⃝2014 Association for Computational Linguistics Information Extraction over Structured Data: Question Answering with Freebase Xuchen Yao 1 and Benjamin Van Durme 1,2 1Center for Language and Speech Processing 2Human Language Technology Center of Excellence Johns Hopkins University Baltimore, MD, USA Abstract Answering natural language questions us- ing the Freebase knowledge base has re- cently been explored as a platform for ad- vancing the state of the art in open do- main semantic parsing. Those efforts map questions to sophisticated meaning repre- sentations that are then attempted to be matched against viable answer candidates in the knowledge base. Here we show that relatively modest information extrac- tion techniques, when paired with a web- scale corpus, can outperform these sophis- ticated approaches by roughly 34% rela- tive gain. 1 Introduction Question answering (QA) from a knowledge base (KB) has a long history within natural language processing, going back to the 1960s and 1970s, with systems such as Baseball (Green Jr et al., 1961) and Lunar (Woods, 1977). These systems were limited to closed-domains due to a lack of knowledge resources, computing power, and abil- ity to robustly understand natural language. With the recent growth in KBs such as DBPedia (Auer et al., 2007), Freebase (Bollacker et al., 2008) and Yago2 (Hoffart et al., 2011), it has be- come more practical to consider answering ques- tions across wider domains, with commercial sys- tems including Google Now, based on Google’s Knowledge Graph, and Facebook Graph Search, based on social network connections. The AI community has tended to approach this problem with a focus on first understanding the in- tent of the question, via shallow or deep forms of semantic parsing (c.f. §3 for a discussion). Typ- ically questions are converted into some mean- ing representation (e.g., the lambda calculus), then mapped to database queries. Performance is thus bounded by the accuracy of the original seman- tic parsing, and the well-formedness of resultant database queries.1 The Information Extraction (IE) community ap- proaches QA differently: first performing rela- tively coarse information retrieval as a way to triage the set of possible answer candidates, and only then attempting to perform deeper analysis. Researchers in semantic parsing have recently explored QA over Freebase as a way of moving beyond closed domains such as GeoQuery (Tang and Mooney, 2001). While making semantic pars- ing more robust is a laudable goal, here we provide a more rigorous IE baseline against which those efforts should be compared: we show that “tradi- tional” IE methodology can significantly outper- form prior state-of-the-art as reported in the se- mantic parsing literature, with a relative gain of 34% F1 as compared to Berant et al. (2013). 2 Approach We will view a KB as an interlinked collection of “topics”. When given a question about one or sev- eral topics, we can select a “view” of the KB con- cerning only involved topics, then inspect every …

Content truncated to 500 words