Know your audience: specializing grounded language models with listener subtraction

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3884–3911 May 2-6, 2023 ©2023 Association for Computational Linguistics Know your audience: specializing grounded language models with listener subtraction Aaditya K Singh Gatsby Computational Neuroscience Unit University College London London, W1T 4JG aaditya.singh.21@ucl.ac.uk David Ding DeepMind London, UK fding@deepmind.com Andrew Saxe Gatsby Computational Neuroscience Unit University College London London, W1T 4JG a.saxe@ucl.ac.uk Felix Hill DeepMind London, UK felixhill@deepmind.com Andrew Kyle Lampinen DeepMind London, UK lampinen@deepmind.com Abstract Effective communication requires adapting to the idiosyncrasies of each communicative context—such as the common ground shared with each partner. Humans demonstrate this ability to specialize to their audience in many contexts, such as the popular game Dixit. We take inspiration from Dixit to formulate a multi- agent image reference game where a (trained) speaker model is rewarded for describing a tar- get image such that one (pretrained) listener model can correctly identify it among distrac- tors, but another listener cannot. To adapt, the speaker must exploit differences in the knowl- edge it shares with the different listeners. We show that finetuning an attention-based adapter between a CLIP vision encoder and a large lan- guage model in this contrastive, multi-agent setting gives rise to context-dependent natu- ral language specialization from rewards only, without direct supervision. Through controlled experiments, we show that training a speaker with two listeners that perceive differently, us- ing our method, allows the speaker to adapt to the idiosyncracies of the listeners. Furthermore, we show zero-shot transfer of the specialization to real-world data. Our experiments demon- strate a method for specializing grounded lan- guage models without direct supervision and highlight the interesting research challenges posed by complex multi-agent communication. 1 Introduction Human language use is communicative, and thus involves substantial adaptation to each conversa- tional partner and context (Clark and Wilkes-Gibbs, 1986; Clark, 1996; Gallois et al., 2005; Frank and Goodman, 2012; Hawkins et al., 2019, 2022). We can adapt our speech to complex social settings, with multiple partners (Mankewitz et al., 2021; Boyce et al., 2022) and competing constraints, such as politeness vs. explicitness (Yoon et al., 2016). While current language and captioning models can increasingly imitate human language and scene de- scriptions (Brown et al., 2020; Tsimpoukelli et al., 2021; Mokady et al., 2021; Alayrac et al., 2022; Elhagry and Kadaoui, 2021), they generally are not explicitly adapted to a particular partner or to satisfy multiple constraints. Common approaches to specializing the language generated by these models fall into two categories: finetuning on a supervised dataset (e.g. Hu et al., 2021), which re- quires task-specific labeled examples, or prompt engineering, which is often brittle and may still require task-specific examples (Liu et al., 2021). Here, we take inspiration from how humans adapt their language to their audience to offer an alter- native approach to specializing grounded language models without direct supervision. Our approach to specialization is inspired by the popular multi-player game Dixit (Roubira, 2008), which rewards each player’s ability to adapt their communication …

Content truncated to 500 words