Learning to Mediate Disparities Towards Pragmatic Communication

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 2829 - 2842 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Learning to Mediate Disparities Towards Pragmatic Communication Yuwei Bao† Sayan Ghosh§∗ Joyce Chai† †Computer Science and Engineering, University of Michigan §ASAPP {yuweibao, sayghosh, chaijy}@umich.edu Abstract Human communication is a collaborative pro- cess. Speakers, on top of conveying their own intent, adjust the content and language expres- sions by taking the listeners into account, in- cluding their knowledge background, personal- ities, and physical capabilities. Towards build- ing AI agents with similar abilities in lan- guage communication, we propose Pragmatic Rational Speaker (PRS), a framework extend- ing Rational Speech Act (RSA). The PRS at- tempts to learn the speaker-listener disparity and adjust the speech accordingly, by adding a light-weighted disparity adjustment layer into working memory on top of speaker’s long-term memory system. By fixing the long-term mem- ory, the PRS only needs to update its working memory to learn and adapt to different types of listeners. To validate our framework, we create a dataset that simulates different types of speaker-listener disparities in the context of referential games. Our empirical results demonstrate that the PRS is able to shift its out- put towards the language that listeners are able to understand, significantly improve the collab- orative task outcome. 1 Introduction In human communication, speakers often adjust their language production by taking into consid- eration listeners’ personality, background knowl- edge, perceptual or physical capabilities etc (Clark, 1996). Recent years have seen an increasing amount of work that explores pragmatic reason- ing based on Rational Speech Act (RSA) (Andreas and Klein, 2016; Fried et al., 2018a,b; White et al., 2020; Cohn-Gordon et al., 2018), multi-agent emer- gent communication framework (Lazaridou et al., 2020; Lazaridou and Baroni, 2020), and Theory of Mind in communication (Bara et al., 2021; Zhu et al., 2021). However, except for (Zhu et al., 2021), ∗Work done during undergraduate study at the University of Michigan. (a) Target (b) Distractor . Literal Speaker: There is an owl on the table. Rational Speaker: There is a pizza on the table. Listener’s Disparity: understands hypernym of food only. Pragmatic Rational Speaker: There is food on the table. Figure 1: TASK: Given two images, the speaker generates a description for the target image and asks the listener to pick out the image described. Both players win if the listener picks the correct one. In this example, a Literal Speaker could generate multiple captions that suit the target, such as the one above, whereas a Rational Speaker limits the description to the unique features of the target (e.g. pizza). If the listener only understands the hypernym of food (disparity), a Pragmatic Rational Speaker would learn the disparity and use food instead of pizza to help the listener understand. most previous works assume that the listeners and the speakers have the same background knowledge and capabilities, including vocabulary size, visual access, and relative locations. This assumption is a great simplification …

Content truncated to 500 words