Inquisitive Question Generation for High Level Text Comprehension

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 6544–6555, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 6544 Inquisitive Question Generation for High Level Text Comprehension Wei-Jen Ko1 Te-Yuan Chen2 Yiyan Huang2 Greg Durrett1 Junyi Jessy Li3 1 Department of Computer Science 2 School of Information 3 Department of Linguistics The University of Texas at Austin {wjko, yuan0061, huang.yiyan}@utexas.edu, gdurrett@cs.utexas.edu,jessy@austin.utexas.edu Abstract Inquisitive probing questions come naturally to humans in a variety of settings, but is a chal- lenging task for automatic systems. One nat- ural type of question to ask tries to fill a gap in knowledge during text comprehension, like reading a news article: we might ask about background information, deeper reasons be- hind things occurring, or more. Despite recent progress with data-driven approaches, generat- ing such questions is beyond the range of mod- els trained on existing datasets. We introduce INQUISITIVE, a dataset of ∼19K questions that are elicited while a person is reading through a document. Compared to existing datasets, INQUISITIVE questions tar- get more towards high-level (semantic and dis- course) comprehension of text. We show that readers engage in a series of pragmatic strate- gies to seek information. Finally, we evalu- ate question generation models based on GPT- 2 (Radford et al., 2019) and show that our model is able to generate reasonable questions although the task is challenging, and highlight the importance of context to generate INQUIS- ITIVE questions. 1 Introduction The ability to generate meaningful, inquisitive ques- tions is natural to humans. Studies among chil- dren (Jirout, 2011) showed that questions serving to better understand natural language text are an organic reflection of curiosity, which “arise from the perception of a gap in knowledge or understand- ing” (Loewenstein, 1994). Because of its promi- nence in human cognition and behavior, being able to formulate the right question is highly sought af- ter in intelligent systems, to reflect the ability to un- derstand language, to gather new information, and to engage with users (Vanderwende, 2007, 2008; Piwek and Boyer, 2012; Rus et al., 2010; Huang et al., 2017). A recent line of work on data-driven This gritty city of 7.6 million rarely gets respect. It often ranks low in “livability” surveys and near the top of the hemisphere’s ugliest capitals. What constitutes "livability"? DEFINITIONAL Why is it referred to as gritty? WHY/CAUSAL Why does it rarely get respect? WHY/CAUSAL Which are the other ones? INSTANTIATION Inquisitive reader Figure 1: Example questions in INQUISITIVE reflecting a range of information-seeking strategies. question generation techniques (Zhou et al., 2017; Yuan et al., 2017; Song et al., 2018; Zhao et al., 2018) focuses on generating questions for datasets like SQuAD (Rajpurkar et al., 2016, 2018). How- ever, factoid questions generated with an answer in mind after the user has read the full text look very different from more natural questions users might have (Kwiatkowski et al., 2019). This has led to work on “answer-agnostic” question generation (Du et al., 2017; Subramanian et al., 2018; Scialom …

Content truncated to 500 words