Controllable Open-ended Question Generation with A New Question Type Ontology

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 6424–6439 August 1–6, 2021. ©2021 Association for Computational Linguistics 6424 Controllable Open-ended Question Generation with A New Question Type Ontology Shuyang Cao and Lu Wang Computer Science and Engineering University of Michigan Ann Arbor, MI {caoshuy, wangluxy}@umich.edu Abstract We investigate the less-explored task of gener- ating open-ended questions that are typically answered by multiple sentences. We first de- fine a new question type ontology which differ- entiates the nuanced nature of questions bet- ter than widely used question words. A new dataset with 4, 959 questions is labeled based on the new ontology. We then propose a novel question type-aware question generation framework, augmented by a semantic graph representation, to jointly predict question fo- cuses and produce the question. Based on this framework, we further use both exemplars and automatically generated templates to improve controllability and diversity. Experiments on two newly collected large-scale datasets show that our model improves question quality over competitive comparisons based on automatic metrics. Human judges also rate our model outputs highly in answerability, coverage of scope, and overall quality. Finally, our model variants with templates can produce questions with enhanced controllability and diversity. 1 Introduction Question-asking has long served as an effective in- strument for knowledge learning (Andre, 1979; To- bin, 1990) and assessing learning progress (Holme, 2003; Downing and Yudkowsky, 2009; Livingston, 2009). Compared to the widely studied task of gen- erating factoid questions that inquire about “one bit” of information (Du et al., 2017; Duan et al., 2017; Li et al., 2019), this work is interested in generating open-ended questions that require deep comprehension and long-form answers (Labutov et al., 2015). Such open-ended questions are valu- able in education, e.g., to facilitate complex knowl- edge acquisition (Lai et al., 2017) and nurture rea- soning skills (Shapley, 2000), as well as in other applications like improving search engines (Han Input: It’s a difficult task to undertake. Teenagers tend to identify gangs with “fitting” in. Peer pressure plays a large part in it and sometimes teenagers have problems with their own identity being part of a gang deals with those issues. It also provides a little bit of respect on the street ... BART SAMPLING: - How do you stop a teen from joining a gang? (PROCEDURAL) - How do you get teenagers to stop being in gangs? (PROCEDURAL) - How do you get teens out of gangs? (PROCEDURAL) BART + QWORD: - How do you get a teenager out of a gang? (PROCEDURAL) - What is the best way to get teenagers out of gangs? (PROCEDURAL) - Why do teenagers join gangs? (CAUSE) TPLGEN: - How do I get [NP] to quit being in [NP]? ⇒ How do I get my son to quit being in a gang? (PROCEDURAL) - What are [NP]? ⇒ What are some programs for teenagers involved in gangs? (EXAMPLE) - Why do [NP] [V] [NP]? ⇒ …

Content truncated to 500 words