Looking for Confirmations: An Effective and Human-Like Visual Dialogue Strategy

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9330–9338 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 9330 Looking for Confirmations: An Effective and Human-Like Visual Dialogue Strategy Alberto Testoni DISI - University of Trento Trento - Italy alberto.testoni@unitn.it Raffaella Bernardi CIMeC and DISI - University of Trento Rovereto (TN) - Italy raffaella.bernardi@unitn.it Abstract Generating goal-oriented questions in Visual Dialogue tasks is a challenging and long- standing problem. State-Of-The-Art systems are shown to generate questions that, although grammatically correct, often lack an effective strategy and sound unnatural to humans. In- spired by the cognitive literature on informa- tion search and cross-situational word learn- ing, we design Confirm-it, a model based on a beam search re-ranking algorithm that guides an effective goal-oriented strategy by asking questions that confirm the model’s conjecture about the referent. We take the GuessWhat?! game as a case-study. We show that dialogues generated by Confirm-it are more natural and effective than beam search decoding without re-ranking. 1 Introduction Quite important progress has been made on multimodal conversational systems thanks to the introduction of the Encoder-Decoder frame- work (Sutskever et al., 2014). The success of these systems can be measured by evaluating them on task-oriented referential games. Despite the high task-success achieved and the apparent linguistic well-formedness of the single questions, the quality of the generated dialogues, according to surface- level features, have been shown to be poor; this holds for systems based on both greedy and beam search (e.g. Shekhar et al. (2019); Zarrieß and Schlangen (2018); Murahari et al. (2019)). Testoni and Bernardi (2021a) found that when taking these surface-level features as a proxy of linguistic qual- ity, the latter does not correlate with task success, and the authors point to the importance of study- ing deeper features of dialogue structures. We aim to develop a multimodal model able to generate dialogues that resemble human dialogue strategies. Cognitive studies show that humans do not al- ways act as “rational” agents. When referring to ob- A B C D 0 .50 .75 .25 A B C D Turn 1: is it living? Yes 1 a) is it in the middle? b) is it a dog? c) is it a person? Turn 2a: is it living? Yes is it a dog? Yes A B C D Turn 2b: is it living? Yes is it a dog? No A B C D Figure 1: At turn 2, among the questions proposed by the beam search, Confirm-it chooses b since it is the most suitable one to confirm the current conjecture. jects, they tend to be overspecific and prefer prop- erties irrespectively of their utility for identifying the referent (Gatt et al., 2013); when searching for information or when learning a language, they tend to follow confirmation-driven strategies. Modelling such behaviour in language learning, Medina et al. (2011) and Trueswell et al. (2013) propose a proce- dure in which a single hypothesized word-referent pair is maintained across learning instances, and it is abandoned only if …

Content truncated to 500 words