Generating Factoid Questions With Recurrent Neural Networks: The 30M Factoid Question-Answer Corpus

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 588–598, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Generating Factoid Questions With Recurrent Neural Networks: The 30M Factoid Question-Answer Corpus Iulian Vlad Serban∗◦ University of Montreal 2920 chemin de la Tour, Montr´eal, QC, Canada Alberto Garc´ıa-Dur´an∗⋄ Universit´e de Technologie de Compi`egne CNRS Rue du Dr Schweitzer, Compigne, France Caglar Gulcehre◦ University of Montreal 2920 chemin de la Tour, Montr´eal, QC, Canada Sungjin Ahn◦ University of Montreal 2920 chemin de la Tour, Montr´eal, QC, Canada Sarath Chandar◦ University of Montreal 2920 chemin de la Tour, Montr´eal, QC, Canada Aaron Courville◦ University of Montreal 2920 chemin de la Tour, Montr´eal, QC, Canada Yoshua Bengio†◦ University of Montreal 2920 chemin de la Tour, Montr´eal, QC, Canada Abstract Over the past decade, large-scale super- vised learning corpora have enabled ma- chine learning researchers to make sub- stantial advances. However, to this date, there are no large-scale question- answer corpora available. In this paper we present the 30M Factoid Question- Answer Corpus, an enormous question- answer pair corpus produced by apply- ing a novel neural network architecture on the knowledge base Freebase to trans- duce facts into natural language ques- tions. The produced question-answer pairs are evaluated both by human evaluators and using automatic evaluation metrics, including well-established machine trans- lation and sentence similarity metrics. Across all evaluation criteria the question- generation model outperforms the compet- ing template-based baseline. Furthermore, when presented to human evaluators, the generated questions appear to be compa- rable in quality to real human-generated questions. * First authors. ◦ Email: {iulian.vlad.serban,caglar.gulcehre, sungjin.ahn,sarath.chandar.anbil.parthipan, aaron.courville,yoshua.bengio}@umontreal.ca ⋄ Email: alberto.garcia-duran@utc.fr † CIFAR Senior Fellow 1 Introduction A major obstacle for training question-answering (QA) systems has been due to the lack of labeled data. The question answering field has focused on building QA systems based on traditional in- formation retrieval procedures (Lopez et al., 2011; Dumais et al., 2002; Voorhees and Tice, 2000). More recently, researchers have started to utilize large-scale knowledge bases (KBs) (Lopez et al., 2011), such as Freebase (Bollacker et al., 2008), WikiData (Vrandeˇci´c and Kr¨otzsch, 2014) and Cyc (Lenat and Guha, 1989).1 Bootstrapping QA systems with such structured knowledge is clearly beneficial, but it is unlikely alone to overcome the lack of labeled data. To take into account the rich and complex nature of human language, such as paraphrases and ambiguity, it would appear that labeled question and answer pairs are necessary. The need for such labeled pairs is even more criti- cal for training neural network-based QA systems, where researchers until now have relied mainly on hand-crafted rules and heuristics to synthesize ar- tificial QA corpora (Bordes et al., 2014; Bordes et al., 2015). Motivated by these recent developments, in this paper we focus on generating questions based on the Freebase KB. We frame question generation as a transduction problem starting from a Freebase fact, represented by a triple consisting of a sub- ject, a relationship and an object, which is trans- 1Freebase is now …

Content truncated to 500 words