Identifying Where to Focus in Reading Comprehension for Neural Question Generation

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2067–2073 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Identifying Where to Focus in Reading Comprehension for Neural Question Generation Xinya Du and Claire Cardie Department of Computer Science Cornell University Ithaca, NY, 14853, USA {xdu, cardie}@cs.cornell.edu Abstract A first step in the task of automatically generating questions for testing reading comprehension is to identify question- worthy sentences, i.e. sentences in a text passage that humans find it worthwhile to ask questions about. We propose a hierar- chical neural sentence-level sequence tag- ging model for this task, which existing approaches to question generation have ig- nored. The approach is fully data-driven — with no sophisticated NLP pipelines or any hand-crafted rules/features — and compares favorably to a number of base- lines when evaluated on the SQuAD data set. When incorporated into an existing neural question generation system, the re- sulting end-to-end system achieves state- of-the-art performance for paragraph-level question generation for reading compre- hension. 1 Introduction and Related Work Automatically generating questions for test- ing reading comprehension is a challenging task (Mannem et al., 2010; Rus et al., 2010). First and foremost, the question generation system must determine which concepts in the associated text passage are important, i.e. are worth asking a ques- tion about. The little previous work that exists in this area currently circumvents this critical step in passage- level question generation by assuming that such sentences have already been identified. In par- ticular, prior work focuses almost exclusively on sentence-level question generation: given a text passage, assume that all sentences contain a question-worthy concept and generate one or more questions for each (Heilman and Smith, 2010; Du et al., 2017; Zhou et al., 2017). In contrast, we study the task of passage-level question generation (QG). Inspired by the large body of research in text summarization on iden- tifying sentences that contain “summary-worthy” content (e.g. Mihalcea (2005), Berg-Kirkpatrick et al. (2011), Yang et al. (2017)), we develop a method to identify the question-worthy sen- tences in each paragraph of a reading compre- hension passage. Inspired further by the success of neural sequence models for many natural lan- guage processing tasks (e.g. named entity recog- nition (Collobert et al., 2011), sentiment classi- fication (Socher et al., 2013), machine transla- tion (Sutskever et al., 2014), dependency pars- ing (Chen and Manning, 2014)), including very re- cently document-level text summarization (Cheng and Lapata, 2016), we propose a hierarchical neural sentence-level sequence tagging model for question-worthy sentence identification. We employ the SQuAD reading comprehen- sion data set (Rajpurkar et al., 2016) for evalua- tion and show that our sentence selection approach compares favorably to a number of baselines in- cluding the feature-rich sentence selection model of Cheng and Lapata (2016) proposed in the con- text of extract-based summarization, and the con- volutional neural network model of Kim (2014) that achieves state-of-the-art results on a variety of sentence classification tasks. We also incorporate our sentence selection com- ponent …

Content truncated to 500 words