Natural Language Does Not Emerge ‘Naturally’ in Multi-Agent Dialog

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2962–2967 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Natural Language Does Not Emerge ‘Naturally’ in Multi-Agent Dialog Satwik Kottur1 and José M.F. Moura1 and Stefan Lee2,3 and Dhruv Batra3,4 1Carnegie Mellon University, 2Virginia Tech, 3Georgia Tech, 4Facebook AI Research Abstract A number of recent works have pro- posed techniques for end-to-end learning of communication protocols among coop- erative multi-agent populations, and have simultaneously found the emergence of grounded human-interpretable language in the protocols developed by the agents, learned without any human supervision! In this paper, using a Task & Talk reference game between two agents as a testbed, we present a sequence of ‘negative’ results culminating in a ‘positive’ one – showing that while most agent-invented languages are effective (i.e. achieve near-perfect task rewards), they are decidedly not inter- pretable or compositional. In essence, we find that natural language does not emerge ‘naturally’, despite the semblance of ease of natural-language-emergence that one may gather from recent literature. We discuss how it is possible to coax the invented languages to become more and more human-like and compositional by in- creasing restrictions on how two agents may communicate. 1 Introduction One fundamental goal of artificial intelligence (AI) is the development of goal-driven dialog agents – specifically, agents that can perceive their environment (through vision, audition, or other sensors), and communicate with humans or other agents in natural language towards a goal. While historically such agents have been based on slot filling (Lemon et al., 2006), the domi- nant paradigm today is neural dialog models (Bor- des and Weston, 2016; Weston, 2016; Serban et al., 2016a,b) trained on large quantities of data. Perhaps somewhat counterintuitively, this current paradigm treats dialog as a static supervised learn- ing problem, rather than as the interactive agent learning problem that it naturally is. Specifi- cally, a typical pipeline is to collect a large dataset of human-human dialog (Lowe et al., 2015; Das et al., 2017a; de Vries et al., 2017; Mostafazadeh et al., 2017), inject a machine in the middle of a di- alog from the dataset, and supervise it to mimic the human response. While this teaches the agent cor- relations between symbols, it does not convey the functional meaning of language, grounding (map- ping physical concepts to words), compositional- ity (combining knowledge of simpler concepts to describe richer concepts), or aspects of planning (why are we having this conversation?). An alternative paradigm that has a long history (Winograd, 1971; Kirby et al., 2014) and is wit- nessing a recent resurgence (Wang et al., 2016; Foerster et al., 2016; Sukhbaatar et al., 2016; Jorge et al., 2016; Lazaridou et al., 2017; Havrylov and Titov, 2017; Mordatch and Abbeel, 2017; Das et al., 2017b) – is situated language learning. A number of recent works have proposed reinforce- ment learning techniques for learning the com- munication protocols of agents situated in virtual environments in a completely end-to-end man- ner – from perceptual …

Content truncated to 500 words