Miss Tools and Mr Fruit: Emergent Communication in Agents Learning about Object Affordances

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3909–3918 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 3909 Miss Tools and Mr Fruit: Emergent communication in agents learning about object affordances Diane Bouchacourt1 and Marco Baroni1,2 1Facebook A.I. Research 2ICREA {dianeb,mbaroni}@fb.com Abstract Recent research studies communication emer- gence in communities of deep network agents assigned a joint task, hoping to gain insights on human language evolution. We propose here a new task capturing crucial aspects of the human environment, such as natural ob- ject affordances, and of human conversation, such as full symmetry among the participants. By conducting a thorough pragmatic and se- mantic analysis of the emergent protocol, we show that the agents solve the shared task through genuine bilateral, referential commu- nication. However, the agents develop mul- tiple idiolects, which makes us conclude that full symmetry is not a sufficient condition for a common language to emerge. 1 Introduction The advent of powerful deep learning architec- tures has revived research in simulations of lan- guage emergence among computational agents that must communicate to accomplish a task (e.g., Jorge et al., 2016; Havrylov and Titov, 2017; Kot- tur et al., 2017; Lazaridou et al., 2017; Lee et al., 2017; Choi et al., 2018; Evtimova et al., 2018; Lazaridou et al., 2018). The nature of the emer- gent communication code should provide insights on questions such as to what extent compara- ble functional pressures could have shaped hu- man language, and whether deep learning models can develop human-like linguistic skills. For such inquiries to be meaningful, the designed setup should reflect as many aspects of human commu- nication as possible. Moreover, appropriate tools should be applied to the analysis of emergent com- munication, since, as several recent studies have shown, agents might succeed at a task without truly relying on their communicative channel, or by means of ad-hoc communication techniques overfitting their environment (Kottur et al., 2017; Bouchacourt and Baroni, 2018; Lowe et al., 2019). We contribute on both fronts. We introduce a game meeting many desiderata for a natural communication environment. We further propose a two-pronged analysis of emerging communica- tion, at the pragmatic and semantic levels. At the pragmatic level, we study communicative acts from a functional perspective, measuring whether the messages produced by an agent have an im- pact on the subsequent behaviour of the other. At the semantic level, we decode which aspects of the extra-linguistic context the agents refer to, and how such reference acts differ between agents. Some of our conclusions are positive. Not only do the agents solve the shared task, but genuine bi- lateral communication helps them to reach higher reward. Moreover, their referential acts are mean- ingful given the task, carrying the semantics of their input. However, we also find that even per- fectly symmetric agents converge to distinct idi- olects instead of developing a single, shared code. 2 The fruit and tools game Our game, inspired by Tomasello’s (2014) conjec- …

Content truncated to 500 words