Question Answering on Freebase via Relation Extraction and Textual Evidence

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 2326–2336, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Question Answering on Freebase via Relation Extraction and Textual Evidence Kun Xu1, Siva Reddy2, Yansong Feng1,∗, Songfang Huang3 and Dongyan Zhao1 1Institute of Computer Science & Technology, Peking University, Beijing, China 2School of Informatics, University of Edinburgh, UK 3IBM China Research Lab, Beijing, China {xukun, fengyansong, zhaody}@pku.edu.cn siva.reddy@ed.ac.uk huangsf@cn.ibm.com Abstract Existing knowledge-based question an- swering systems often rely on small an- notated training data. While shallow meth- ods like relation extraction are robust to data scarcity, they are less expressive than the deep meaning representation methods like semantic parsing, thereby failing at an- swering questions involving multiple con- straints. Here we alleviate this problem by empowering a relation extraction method with additional evidence from Wikipedia. We first present a neural network based re- lation extractor to retrieve the candidate answers from Freebase, and then infer over Wikipedia to validate these answers. Ex- periments on the WebQuestions question answering dataset show that our method achieves an F1 of 53.3%, a substantial im- provement over the state-of-the-art. 1 Introduction Since the advent of large structured knowledge bases (KBs) like Freebase (Bollacker et al., 2008), YAGO (Suchanek et al., 2007) and DBpedia (Auer et al., 2007), answering natural language questions using those structured KBs, also known as KB- based question answering (or KB-QA), is attract- ing increasing research efforts from both natural language processing and information retrieval com- munities. The state-of-the-art methods for this task can be roughly categorized into two streams. The first is based on semantic parsing (Berant et al., 2013; Kwiatkowski et al., 2013), which typically learns a grammar that can parse natural language to a so- phisticated meaning representation language. But such sophistication requires a lot of annotated train- ing examples that contains compositional struc- tures, a practically impossible solution for large KBs such as Freebase. Furthermore, mismatches between grammar predicted structures and KB structure is also a common problem (Kwiatkowski et al., 2013; Berant and Liang, 2014; Reddy et al., 2014). On the other hand, instead of building a for- mal meaning representation, information extraction methods retrieve a set of candidate answers from KB using relation extraction (Yao and Van Durme, 2014; Yih et al., 2014; Yao, 2015; Bast and Hauss- mann, 2015) or distributed representations (Bordes et al., 2014; Dong et al., 2015). Designing large training datasets for these methods is relatively easy (Yao and Van Durme, 2014; Bordes et al., 2015; Serban et al., 2016). These methods are often good at producing an answer irrespective of their correct- ness. However, handling compositional questions that involve multiple entities and relations, still re- mains a challenge. Consider the question what mountain is the highest in north america. Relation extraction methods typically answer with all the mountains in North America because of the lack of sophisticated representation for the mathematical function highest. To select the correct answer, one has to retrieve all the heights of …

Content truncated to 500 words