“What makes a question inquisitive?” A Study on Type-Controlled Inquisitive Question Generation

Proceedings of the 11th Joint Conference on Lexical and Computational Semantics, pages 240 - 257 July 14-15, 2022 ©2022 Association for Computational Linguistics “What makes a question inquisitive?” A Study on Type-Controlled Inquisitive Question Generation Lingyu Gao∗1, Debanjan Ghosh2, and Kevin Gimpel1 1Toyota Technological Institute at Chicago 2Educational Testing Service {lygao, kgimpel}@ttic.edu, dghosh@ets.org Abstract We propose a type-controlled framework for inquisitive question generation. We annotate an inquisitive question dataset with question types, train question type classifiers, and fine- tune models for type-controlled question gen- eration. Empirical results demonstrate that we can generate a variety of questions that ad- here to specific types while drawing from the source texts. We also investigate strategies for selecting a single question from a generated set, considering both an informative vs. inquis- itive question classifier and a pairwise ranker trained from a small set of expert annotations. Question selection using the pairwise ranker yields strong results in automatic and manual evaluation. Our human evaluation assesses mul- tiple aspects of the generated questions, finding that the ranker chooses questions with the best syntax (4.59), semantics (4.37), and inquisitive- ness (3.92) on a scale of 1-5, even rivaling the performance of human-written questions. 1 Introduction Recently, interest has grown in the task of auto- matic question generation (AQG) from text (Sun et al., 2018; Kumar and Black, 2020). AQG is use- ful in building conversational AI systems (Bordes et al., 2017; Gao et al., 2019), generating synthetic examples for QA (Alberti et al., 2019; Dong et al., 2019; Sultan et al., 2020), and educational applica- tions, such as intelligent tutoring and instructional games (Chen et al., 2018; Flor and Riordan, 2018). In the majority of such studies, AQG focuses on generating factual questions that tend to ask about specific information in the text (i.e., “who did what to whom”) (Du et al., 2017). Instead of asking factual questions with answers already present in the text, Ko et al. (2020) argued that human readers instinctively ask questions that are curiosity-driven, answer-agnostic, and seek a ∗ Partially done as part of an internship at Educational Testing Service. Context ...The plan places an indicated value on the real estate operation, Santa Fe Pacific Realty Corp., of $ 2 billion. Source sentence Santa Fe Pacific directors are expected to re- view the plan at a meeting today, according to people familiar with the transaction. BASE What kind of meeting? SPAN How will the directors review the plan? Explanation Why are they reviewing the plan? Background Is it expected to review the plan today? Elaboration What will the review entail? Instantiation Which directors are expected to review the plan? Definition what is that? Forward What are the directors expected to review? Informative Who are Santa Fe Pacific directors expected to review? Table 1: Examples of generated questions given the article context and source sentence (with span in bold). high-level understanding of the document being read. They released a dataset of such curiosity- driven questions (henceforth INQUISITIVE; for de- tails, see Section 2). …

Content truncated to 500 words