Paraphrase for Open Question Answering: New Dataset and Methods

Proceedings of 2016 NAACL Human-Computer Question Answering Workshop, pages 53–61, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics Paraphrase for Open Question Answering: New Dataset and Methods Ying Xu†, Pascual Mart´ınez-G´omez‡, Yusuke Miyao§, Randy Goebel† †Department of Computing Science, University of Alberta ‡National Institute of Advanced Industrial Science and Technology (AIST) §National Institute of Informatics / JST, PRESTO †{yx2,rgoebel}@ualberta.ca ‡pascual.mg@aist.go.jp §yusuke@nii.ac.jp Abstract We propose a new open question answer- ing framework for question answering over a knowledge base (KB). Our system uses both a curated KB, Freebase, and one that is ex- tracted automatically by an open information extraction model, IE KB. Our system consists of only one layer of paraphrase, compared to the three layers used in a previous open ques- tion answering system (Fader et al., 2014). However, because of the more accurately ex- tracted relation triples in IE KB, combined with linked entities from IE KB to Freebase, our system achieves a 7% absolute gain in F1 score over the previous system. 1 introduction There are two broad classes of systems that pro- vide question answering (QA) from a knowledge base (KB). One uses semantic parsing (Berant and Liang, 2014; Reddy et al., 2014; Yih et al., 2015), and the other uses information extraction (IE). Se- mantic parsing systems depend on highly accurate knowledge bases such as Freebase, which are accu- rate but incomplete (Riedel et al., 2013; Fader et al., 2014). However, although semantic parsing systems currently achieve higher performance than IE-based systems, we think it is desirable to continue to de- velop the latter as an alternative or in combination with the former. One major challenge for question answering from a KB is the many ways in which relations can be ex- pressed. On the one hand, we need to deal with lan- guage variability, for example, acknowledging that the following two questions have the same mean- ing: “What character did Natalie Portman play in Star Wars?” and “What is the role of Natalie Port- man in Star Wars?”. We call this NL-NL paraphras- ing, since it requires the identification of a map be- tween two natural language expressions. On the other hand, we need to bridge the gap between the expression of relations in a curated knowledge base, such as Freebase, and relations conveyed in natu- ral language sentences. We refer to this as NL- KB paraphrasing. For instance, general QA will require a mapping between the natural language relation brother and the Freebase relation “/peo- ple/person/sibling s.” Our contribution to Open IE question answer- ing is three-fold. First, we provide a new wide- coverage store of automatically extracted relation triples, which has a higher precision than that of pre- vious work (Fader et al., 2014). This Open IE triple store allows us to tackle the NL-NL paraphrasing problem. Second, the entities in our dataset is linked to Freebase, which allows us to tackle the NL-KB paraphrasing problem. Third, we propose a simple and effective open QA framework that consists of …

Content truncated to 500 words