FreebaseQA: A New Factoid QA Data Set Matching Trivia-Style Question-Answer Pairs with Freebase

Proceedings of NAACL-HLT 2019, pages 318–323 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 318 FreebaseQA: A New Factoid QA Data Set Matching Trivia-Style Question-Answer Pairs with Freebase Kelvin Jiang∗ † and Dekun Wu∗ and Hui Jiang Department of Electrical Engineering and Computer Science York University, 4700 Keele Street, Toronto, Ontario, M3J 1P3, Canada {kelvin,jackwu,hj}@eecs.yorku.ca Abstract In this paper, we present a new data set, named FreebaseQA, for open-domain factoid question answering (QA) tasks over struc- tured knowledge bases, like Freebase. The data set is generated by matching trivia-type question-answer pairs with subject-predicate- object triples in Freebase. For each collected question-answer pair, we first tag all entities in each question and search for relevant predi- cates that bridge a tagged entity with the an- swer in Freebase. Finally, human annota- tion is used to remove false positives in these matched triples. Using this method, we are able to efficiently generate over 54K matches from about 28K unique questions with mini- mal cost. Our analysis shows that this data set is suitable for model training in factoid QA tasks since FreebaseQA provides more lin- guistically sophisticated questions than other existing data sets. The data set is available for free download at http://github.com/ infinitecold/FreebaseQA. 1 Introduction Within the field of natural language processing (NLP), there has been an increase in developments towards various real-world applications, such as factoid question answering (QA): the process of obtaining the answer(s) to a factual question - sim- ilar to trivia game settings. For this task to be suc- cessfully completed, there are several steps that need to occur. Notably, we need to interpret and parse the question, determine the domain of rele- vance, eliminate ambiguities if existent, and pin- point the exact answer to the question asked. For- tunately, this task has been simplified with the emergence of large knowledge graphs, including Freebase (Bollacker et al., 2008), from where we ∗Equal contribution. †Currently at University of Waterloo. Work was done at York University. can retrieve information. Knowledge graphs are colossal networks of data that describe concepts, entities, and their relations. In fact, Freebase is the largest publicly-available knowledge graph, con- sisting of 4 million nodes and approximately 3 billion edges (Google, 2017). Each node repre- sents an entity existing in the physical world, such as a person, a location, or an organization. Each edge represents a relation between two entities, in a directed manner from a subject node to an ob- ject node. In Freebase, these edges are referred to as predicates, and a collection of a subject- predicate-object is referred to as a triple. An ex- ample triple in Freebase is the subject Clarissa, predicate book.written work.author and object Samuel Richardson, explaining that the book Clarissa is written by author Samuel Richardson. Specifically, we take advantage of these relations between entities, which describe facts, to help with factoid question answering. We believe open-domain factoid QA over structured knowledge graphs like Freebase is a very interest- ing NLP task since it …

Content truncated to 500 words