Comparing Text Representations: A Theory-Driven Approach

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 563–570 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 563 Open-domain clarification question generation without question examples Julia White Electrical Engineering Stanford University Gabriel Poesia Computer Science Stanford University Robert Hawkins Psychology Princeton University Dorsa Sadigh Electrical Engineering, Computer Science Stanford University Noah Goodman Computer Science, Psychology Stanford University Abstract An overarching goal of natural language pro- cessing is to enable machines to communicate seamlessly with humans. However, natural language can be ambiguous or unclear. In cases of uncertainty, humans engage in an in- teractive process known as repair: asking ques- tions and seeking clarification until their uncer- tainty is resolved. We propose a framework for building a visually grounded question- asking model capable of producing polar (yes- no) clarification questions to resolve misunder- standings in dialogue. Our model uses an ex- pected information gain objective to derive in- formative questions from an off-the-shelf im- age captioner without requiring any supervised question-answer data. We demonstrate our model’s ability to pose questions that improve communicative success in a goal-oriented 20 questions game with synthetic and human an- swerers. 1 Introduction Human-machine interaction relies on accurate transfer of knowledge from users. However, nat- ural language input can be ambiguous or unclear, giving rise to uncertainty. A fundamental aspect of human communication is collaborative grounding, or seeking and providing incremental evidence of mutual understanding through dialog. Specifically, humans can correct for uncertainty through cooper- ative repair (Clark, 1996; Purver et al., 2002; Arkel et al., 2020) which involves interactively asking questions and seeking clarification. Making and recovering from mistakes collaboratively through question-asking is a key ingredient in grounding meaning and therefore an important feature in di- alog systems (Benotti and Blackburn, 2021). In this work, we focus on the computational chal- lenge of generating clarification questions in visu- ally grounded human-machine interactions. One popular approach is to train an end-to-end model to map visual and linguistic inputs directly Figure 1: Our model takes the role of questioner in a question-driven communication game where it must guess which image is being described by the answerer. The interaction ends with the model returning a guess for which image the answerer is referring to. to questions (Yao et al., 2018; Das et al., 2017). This approach is heavily data-driven, requiring large annotated training sets of questions under different goals and contexts. Another approach has drawn from work on active learning and Optimal Experiment Design (OED) in cognitive science to search for questions that are likely to maximize ex- pected information gain from an imagined answerer (Wang and Lake, 2019; Lee et al., 2018; Misra et al., 2018; Rao and Daumé III, 2018; Rothe et al., 2017; Kovashka and Grauman, 2013). Much of this work has relied on large-scale question-answer datasets (Kumar and Black, 2020; de Vries et al., 2017) for training or retrieval to propose candidate ques- tions or evaluate their expected utility. Others, like (Yu et al., 2020), derive questions from attribute …

Content truncated to 500 words