Ask what’s missing and what’s useful: Improving Clarification Question Generation using Global Knowledge

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4300–4312 June 6–11, 2021. ©2021 Association for Computational Linguistics 4300 Ask what’s missing and what’s useful: Improving Clarification Question Generation using Global Knowledge Bodhisattwa Prasad Majumder†♣ Sudha Rao♦ Michel Galley♦ Julian McAuley♣ ♣Department of Computer Science and Engineering, UC San Diego {bmajumde, jmcauley}@eng.ucsd.edu ♦Microsoft Research, Redmond {sudha.rao, mgalley}@microsoft.com Abstract The ability to generate clarification questions i.e., questions that identify useful missing in- formation in a given context, is important in reducing ambiguity. Humans use previous ex- perience with similar contexts to form a global view and compare it to the given context to ascertain what is missing and what is useful in the context. Inspired by this, we propose a model for clarification question generation where we first identify what is missing by tak- ing a difference between the global and the local view and then train a model to identify what is useful and generate a question about it. Our model outperforms several baselines as judged by both automatic metrics and humans. 1 Introduction An important but under-explored aspect of text un- derstanding is the identification of missing infor- mation in a given context i.e., information that is essential to accomplish an underlying goal but is currently missing from the text. Identifying such missing information can help to reduce ambiguity in a given context which can aid machine learning models in prediction and generation (De Boni and Manandhar, 2003; Stoyanchev et al., 2014). Rao and Daumé III (2018, 2019) recently proposed the task of clarification question generation as a way to identify such missing information in context. They propose a model for this task which while success- ful at generating fluent and relevant questions, still falls short in terms of usefulness and identifying missing information. With the advent of large-scale pretrained generative models (Radford et al., 2019; Lewis et al., 2019; Raffel et al., 2019), generating fluent and coherent text is within reach. However, generating clarification questions requires going beyond fluency and relevance. Doing so requires understanding what is missing, which if included could be useful to the consumer of the information. †Work done during an internship at Microsoft Research TITLE: Sony 18x Optical Zoom 330x Digital Zoom Hi8 Camcorder DESC: Sony Hi-8mm Handycam Vision camcorder 330X digital zoom, Nightshot(TM) Infrared 0 lux system, Special Effects, 2.5" SwivelScreen color LCD and 16:9 recording mode, Laserlink connection. Image Stabilization, remote, built in video light. QUESTION: Can I manually control the video quality? Table 1: Product description from amazon.com paired with a clarification question generated by our model. Humans are naturally good at identifying miss- ing information in a given context. They possibly make use of global knowledge i.e., recollecting pre- vious similar contexts and comparing them to the current one to ascertain what information is miss- ing and if added would be the most useful. Inspired by this, we propose a two-stage framework for the task of clarification question generation. …

Content truncated to 500 words